December 31, 2015

SLES 12 toolchain module available for Linux on z

SUSE has released the toolchain module for Linux on System z. This is the first officially supported gcc compiler that supports the z13.
To install you need to add the product and update repository with "yast2 repositories" and then you can install it with

# zypper install sle-module-toolchain-release
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following 10 NEW packages are going to be installed:
  cpp5 gcc5 gcc5-c++ gcc5-fortran gcc5-locale libgfortran3 libstdc++6-devel-gcc5 patterns-toolchain-gcc5
  sle-module-toolchain-release sle-module-toolchain-release-POOL

The following NEW pattern is going to be installed:

The following NEW product is going to be installed:
  "Toolchain Module"

The following 6 recommended packages were automatically selected:
  cpp5 gcc5-c++ gcc5-fortran gcc5-locale libstdc++6-devel-gcc5 patterns-toolchain-gcc5

10 new packages to install.
Overall download size: 26.2 MiB. Already cached: 0 B  After the operation, additional 136.1 MiB will be
Continue? [y/n/? shows all options] (y): y


As you see you get C, C++ and Fortran. To enable z13 instructions use the -march=z13 option.

December 29, 2015

SUSE Linux Enterprise 12 Service Pack 1 (SLES 12 SP1) released

On December 22nd just in time for Christmas SUSE has released the latest updated to their flag ship server distribution.

The kernel level for 12.1 is kernel-3.12.49-11.1. As one of the top features this distribution is the first one, which allows full exploitation of the z13. And this distribution is so far the only one that's supported by IBM's version of KVM as well as Docker. SUSE has published a nice summary of the the z specific news.
And of course it has many bug fixes. So after the installation be sure to run an update to install the latest fixes from the maintweb. 

Here is my usual summary of links to more information:
A frequently asked question is about the location of the older libstdc++ libraries that are needed for compatibility with older software products. There are two locations:
  • the oldest ones are in a package called compat, that you can install with zypper directly. It has and
  • the libstdc++33 is part of the Legacy Module that SUSE provides. Add it to your repositories with yast2 repositories and then you can install. It contains the
(updated 1/20/2016)

December 18, 2015

Java OpenJDK and Just In Time Compiler (JIT)

Since a while OpenJDK has been available for Linux on the mainframe as an open source Java. In the last week I had two different performance situations with this. The reason is very simple: OpenJDK for z, doesn't have a JIT. So it can be used e.g. for writing an installer but for everything that's repeating the IBM JDK is the only choice.
The effect varies by workload, but it's usually more that one order of magnitude. In the example above showing relative performance for an average workload  the IBM Java does better even without the JIT and with the JIT it's more than a factor of 50! I had to break up the axis to show it in one chart.

December 3, 2015

IBM LinuxONE Community Cloud is open

The Community Cloud announced at LinuxCon is accepting registrations and afterwards you deploy and start your Linux image. Please read the quickstart guide before get going - at least read if you get stuck :-)

If you don't do this, then at least read the invitation email. Here are the top hints:
  • Access is given using a public key - generate it in the Web GUI and then select it for deployment. This is not selected by default and without that key you don't get into the new system! 
  • When you deploy, select the new project assigned to you and not "public" which is the default. You can only work on your project area! 
  • You can only have one Linux server there. So if you want to switch between images, you need to delete the old one first. 
There is also a FAQ available which covers general aspects of this.

Happy testing!

(updated 08/12/2015)

November 23, 2015

Red Hat Enterprise Linux (RHEL) 7.2 released

On November 19th 2015 Red Hat released the next regular update to their flag ship operating system.

The kernel level for 7.2 is kernel-3.10.0-327.el7. The number of technology previews for System z has been greatly reduced, successful testing allows now support for those features.

Here is my usual summary of links to more information:
From a Linux on z Systems perspective this is a major step. This is the first distribution that supports SMT2 natively.So if you install it, you will have twice as many logical CPUs.

And this is the first Red Hat distribution that supports the High Availability and Resilient Storage Add-Ons. So far some restrictions apply, but they hopefully can be lifted in later releases.

(updated 4/26/2016) 

November 12, 2015

Fedora 23 for IBM z Systems released

The new Fedora 23 for z Systems / s390x was released 11/10/2015. Thanks to the Fedora team for making this available! The kernel level is kernel-4.2.3-300.

As usual the download is available from the Fedoraproject site and the respective mirrors and known issues  are covered in the wiki.

October 23, 2015

How long is a mainframe in service?

One of the really nice things about Linux on z is the fact that the underlying hardware has a long life. This means that for complex software solutions that need extensive and expensive testing you can keep the system going over a long time period. The Linux distributions on the mainframe are supporting this model with their long term support contracts.

But how long can you get service? There is a now updated summary presentation out on TechDocs called "IBM Mainframe Life Cycle History" (PPT, PDF). It has a nice overview chart which gives the answer to this question: the average support cycle over the past 21 years has been 11.3 years!

October 22, 2015

IBM Storage Support Matrix (SSIC) updated for Linux on z

This week there has been a major update on the  IBM System Storage Interoperation Center (SSIC). Now in addition to the high end boxes and Flash Systems also many entry and mid range boxes are supported like a V3500, V3700, V5000, V7000. To get a quick view select under connection "FCP (z Systems)":

So for all those systems no SVC is needed any more!

And as a side remark: all the supported tapes are also documented in the SSIC, 

October 12, 2015

Random number generators for Linux on z Systems

For Linux on z Systems there are multiple kernel interfaces to get to (pseudo) random data. First there are the two standard Linux interfaces.
  • /dev/random : blocking interface used for really good random numbers
  • /dev/urandom : nonblocking interface used for everything else
Those interfaces work the same on z Systems as they do on other platforms. However in highly virtualized environments there are really idle servers where nothing is happening. This means that there isn't a lot of entropy generated and therefore /dev/random will block.

On top for z Systems there are two additional interfaces:
  • /dev/prandom : this is a hardware assisted pseudo random number generator using the System z CPACF instructions. To enable it do a modprobe prng. Further details can be found in the "Device Drivers, Features, and Commands" book for upstream, SUSE and Red Hat. In those books search for prng. 
  • /dev/hwrng : this is a true random number generator using the CryptoExpress CCA co-processor function. For this to work, you need the hardware card installed and configured to your Linux. Then a modprobe z90crypt starts it. 
As expected prandom is faster than urandom. But the additional card with the real random number generator is even faster and doesn't use CPU. Obviously when you share the card the throughput will be split between the Linux images.

If your application is written against /dev/random and there isn't enouh entropy in the system, you can refill it by using the rngd daemon. To start it use then command rngd -r /dev/hwrng. Of course you need to install the rng-tools package first. Caveat: you may need to change the service configuration file to point to /dev/hwrng. Or you create the /dev/hwrandom (which is the default for rngd) device node by linking to /dev/hwrng.

September 21, 2015

KVM for IBM z Systems

KVM 1.1.1 (new version) blog post

After the announcement in August, KVM for IBM z Systems is now generally available.  This takes the open source approach for virtualization to the mainframe. 

Here are a few links for further details:
If you encounter any problems, please open a PMR with IBM. 

(updated 12/07/2015) 

September 14, 2015

The Virtualization Cookbook for IBM z Systems - updated

The Redbook "The Virtualization Cookbook for z/VM 6.3, RHEL 6.4 and SLES 11 SP3" and the corresponding code examples got an update to the latest distribution levels. They have also reorganized the material into three different books, which I personally don't like that much - but others say this is the way to go.

August 19, 2015

Minecraft on the Mainframe

LinuxCon video - Joran Siu

Joran Siu from the IBM Java team installed Minecraft Server on the mainframe. In this entertaining talk he shows the results and the optimizations used, some of them only available on zEC12 and z13.

August 18, 2015

Linux on System z - Financial Trading demo from LinuxCon

Donna Dillenberger is showing a pure open source demo on the mainframe combining all the new technologies like Spark, MongoDB, Postgres, MariaDB, Node.js, Docker, Chef etc.
It's using live analytics for streams from Twitter and S&P financial data, showing the power of realtime analysis on the mainframe.
She is also providing a look at the infrastrucure driving this and even shows a fail over from one site to another using Hyperswap.
Towards the end she is providing real performance data for some of the components used.
During this presentation your probably want to pause and playback to get all the information she packed into those 18 minutes. Here is a breakdown (minutes:seconds) for reference:
  • 0:00: demonstration
  • 7:48: a look at the infrastructure
  • 11:22: failover by killing one storage subsystem
  • 13:45: performance data
  • 17:00: agility & summary
(updated 8/19/2015)

August 17, 2015

LinuxCon announcements for Linux on z

Today IBM made some interesting announcements at LinuxCon. I'm going to cover more of the technical stuff in separate posts - this is for the overview.
(updated 2/12/2015)

August 14, 2015

Node.js 1.2 for Linux on z is available on developerworks

IBM has updated the Node.js environment for the mainframe (SDK) to the latest stable community release (version 0.12.7).  You can find it here on developerworks. Besides functional enhancements it also contains an important security update for openssl. The release notes and the updated documentation are also available.

The Linux on z Systems downloads are now also linked from the official download side at right at the bottom of the page.

August 12, 2015

International Oracle on z Systems SIG Conference 2015 - presentations available

The presentations and some demos from the recent conference are availble for download. Besides the general overview presentations, there have been a few more specialized ones this time:
You can also find the presentations from previous years on this site. 

August 11, 2015

Example for using Apache Spark on the mainframe

This YouTube video from Paul DiMarzio shows an interesting analytic example from the Hack Spark contest. Thanks Paul for creating this!

August 10, 2015

z/VM 6.3 achieves common criteria certification at EAL4+ level

Earlier this year z/VM 6.3 including SSI (single system image) has achieved the EAL4+ certification from the German BSI (Bundesamt für Sicherheit in der Informationstechnik).
All z/VM certifications can be found at the "z/VM Security and Integrity Resources" site.

August 7, 2015

Red Hat Enterprise Linux 6.7 released

On July 22, 2015 Red Hat has announced the availability of RHEL 6.7. The kernel level is now kernel-2.6.32-573.el6, for the main bug fixes see the kernel update description.

As usual there are the release notes and the technical notes in two separate documents. The complete documentation can be reached from the Red Hat documentation page (you need to select "6" on the left bar).

For System z this has been a mainly a fix release. Only a few small enhancements have been added all described in the technical notes. Here are some of them:
The IBM documentation for RHEL 6.4 on developerworks still applies for this release.

August 6, 2015

SUSE Linux Enterprise Server 11 Service Pack 4 (SLES 11 SP4 ) released

On July 16th 2015 SUSE has announced the availability of the latest service pack to their SLES11 distribution. This release is a maintenance release, keeping the major kernel version stable. Here is the usual collection of links:

June 24, 2015

Linux on z Docker demo at DockerCon

Since a while there is an experimental version of Docker available for Linux on z. This has now been demoed at DockerCon. Here is the video link: the demo starts around 15:40. Of course they could have run everything on System z and some more visual bells and whistles could have been added fro marketing.
However from a technical point of view it shows the prototype running and integrating with all the cool cloud stuff from Docker :-) 

June 22, 2015

New Whitepaper "z/VM 6.3 HiperDispatch - Polarization Modes and Middleware Performance"

This white paper looks at the effect of using the HiperDispatch feature (introduced with z/VM 6.3) with a mixed Linux workload. It also provides results for applying the scalability APAR VM65586.
 The improvements in the mixed workload including Websphere and Oracle DB shows that this APAR is not only valuable for z13, but rather for everyone with a larger number of IFLs. And if you are still running an old pre 6.3 z/VM release, now is the time to upgrade!

June 11, 2015

Support information for IBM storage products

All the supported combinations for IBM storage can be found in the IBM System Storage Interoperation Center (SSIC).
For Linux on z the best way to get the number of combinations down is to first select "IBM z Systems" as the host platform and then continue with the storage family.

Very important: if your combination isn't listed there, please get a "Request for Price Quotations (RPQ)" submitted by your IBM rep as described on the SSIC start page.

April 14, 2015

Middleware / application testing - operating system, hypervisor and hardware

After the IBM statement of directions to support KVM on System z I've been asked many times for a criteria if a certain ISV software or customer solution needs to be retested or not. As in performance the answer is: it depends. Basically there are two / four different types of applications / products to consider
  1. No retesting needed - certified and tested towards the operating system:
    Applications that use a standard API only, like Java or standard library functions that don't need and exploit the visibility of the underlying (virtualized) hardware.
  2. Retesting recommended - certified and tested towards the operating system:
    Applications that do interact with virtualized hardware to a certain extend e.g. accessing raw devices.
  3. Retesting and some coding needed - certified and tested towards the operating system and the virtualization layer:
    Applications that actively interact with the hypervisor, e.g. by vmcp commands under z/VM. They then needs an equivalent implementation of that kind of functionality for every type of virtualization. Other examles are the use of zLinux hypfs (hypervisor file system) or using the hypervisor details surfaced in /proc/sysinfo.
  4. Retesting and additional code for the hypervisor needed - certified and tested towards the operating system and the virtualization layer:
    Applications that need hooks and special interfaces into the hypervisor e.g. for monitoring or systems management. Every new hypervisor will require new code as e.g. the reported performance metrics and systems management interfaces are different
For Linux on the mainframe today there is only z/VM and LPAR. And z/VM is really virtualizing the hardware in a 1:1 fashion as good as possible (with a lot of firmware and hardware support to keep the overhead to minimal levels).  For such a hardware virtualization the category #2 is nearly not existent. If there is no active interaction with hypervisor, no retesting is needed.
KVM has the concept of new virtual devices with new device names, so some products from #1 are going to slip into category #2.

(updated 4/15/2015)

March 19, 2015

SUSE Linux Enterprise Server 12 (SLES 12) released

In November last year SUSE has announced the availability of their next major distribution release SUSE Linux Enterprise Server (SLES 12). It's an interesting change in strategy that SUSE chose to release it after the competing RHEL 7 distribution. In the past SUSE has always been first to the market.

Here is a the usual list of interesting information links:
SUSE has released the first maintenance web kernel for SLES12  (kernel-default-3.12.36-38.1.s390x.rpm)
It contains quite some fixes that you really want to have on your system.

March 18, 2015


In January 2015 IBM announced the new version of the mainframe called z13.  As of March 9th 2015 the successor of the zEC12 is officially available.

z13 mainframe glas model
z13 glas model - photo Michael Storzer

This machine has many new features that I will cover in separate blog posts. The full advantages from all of them will become available with more and more exploitation and adaptions during the next year.

From a Linux perspective the IBM tested platforms has been updated to include z13.
Please check the minimum and recommended kernel levels to ensure a smooth upgrade to z13!

Also Red Hat and SUSE updated their web sites to include the certifications:
From a z/VM perspective only z/VM 6.2 and z/VM 6.3 is being supported. But you need to install the required service before updating the hardware! Exploitation will only come with z/VM 6.3, so please plan for upgrades! And there is no z/VM 5.4 support any more. 

Warning: Unsupported Linux releases and/or missing z/VM service may lead to virtual server failures.

Other useful links:
For the z13s and z13 GA2 announcements, please refer to this link.  

(updated 02/17/2016)

March 7, 2015

Red Hat Enterprise Linux 7.1 released

On March 5th 2015 Red Hat released the first update of their RHEL 7 operating system. 

The kernel level for 7.1 is kernel-3.10.0-229.el7. The number of technology previews for System z has been greatly reduced, successful testing allows now support for those features.

Here is my usual summary of links to more information:
As this is a brand new release there are some hick ups aka bugs to be expected. If you are hitting one, please open a bug report with Red Hat and/or IBM.

(updated 3/18/2015) 

February 19, 2015

Problem with the latest crypto device driver updates on RHEL 6.6 / SLES 11.3 on older hardware

With the introduction of the code for more crypto domains (required for z13) into the service streams of RHEL 6.6 (kernel 2.6.32-504-3.3 and 2.6.32-504.8.1), SLES 11.3 (kernel 3.0.101-42-1) and SLES12 (kernel 3.12.32-33.1) an unfortunate regression has been introduced. The crypto card support for z196 / z114 and older machines is no longer working with the new code. If you are in that situation, please update to the levels below.

This has been fixed in the following kernel levels.
(updated 4/23/2015)

January 26, 2015

Fedora 21 for IBM System z released

The new Fedora 21 for System z / s390x was released 12/18/2014. Thanks to the Fedora team for this time adding additional packages! 

The download is available from the Fedoraproject site and the respective mirrors and known issues  are covered in the wiki.