SUSE Linux Enterprise 12 Service Pack 3 (SLES 12 SP3) released

z14

Oracle 12c R1 and R2 now certified on RHEL 7.1 and SLES 12 SP1 for Linux on z / LinuxONE

SUSE Linux Enterprise 12 Service Pack 2 (SLES 12 SP2) released

On November 8th 2016 eleven months after the last service pack  SUSE has released the latest updated to their flag ship server distribution. SUSE also has a blog post with some more details.

The kernel level for 12.2 is kernel-4.4.21-69.1. This is a jump in the kernel version from the old 3.12 based kernel. It's the same major kernel level that's being used by Canonical for Ubuntu 16.04. And with that kernel a lot of new functionality is delivered. SUSE and IBM worked hard to ensure that for applications it behaves the same.

What's new (details see documentation and release notes):

• The release notes states that SUSE now fully supports MariaDB. They have version 10.0.27 inlcuded
• Networking:
• For OSA the specification of a portname is no longer needed
• Hipersockets can now be bridged to Erhernet
• Layer2 offloads are enabled
• The 10 GbE RoCE card now also supports the DAPL/OFED stack on
• Encryption and security updates
• Glibc has now the first set of SIMD exploitations for z13
• Toolchain module updates
• Full KVM host support - so you can now run SLES12 as a Hypervisor on System z an Linux boxes 
• ....

Here is my usual summary of links to more information:

• Complete SLES 12 documentation
• SLES 12.2 release notes (s390x version)
• z Systems / s390x specific information 
• release notes for High Availability Extensions (z Systems / s390x version)  
• release notes for SDK (z Systems / s390x version)
• SUSE tuning guide updated to SP2
• Whitepaper on SUSE's module concept
• IBM documentation on developerworks
• Docker support statement

(updated 5/2/2017)

z13 feature exploitation - which distribution levels are required

In this post I'll try to summarize which distribution levels, packages, APARs, PTFs etc are needed to exploit a given z13 (performance) function. I expect this to change over time, so please revisit for updates.

For the virtualization layers I use the label "(guest)" for enablement of guest exploitation. So you need at least the given version of the Hipervisor to exploit a certain function in the guest. Of course you also need the respective guest support.

All version levels in the table are minimum levels.

Feature	Red Hat	SUSE	Ubuntu	z/VM (host)	z/VM (guest)	KVM (host)	KVM (guest)
SMT	7.2	12 SP1	16.04	6.3+		1.1.1
SIMD: kernel support – compiled SIMD apps can run	7.2	12 SP1	16.04		6.3+(3)	1.1.1	1.1.1
z13 tool chain support (compiler, binutils)	tbd	12 SP1	16.04		6.3+		1.1.0
PCI: RoCE 10 Gb Ethernet	7.2(1)	12 SP1	16.04		6.3+
PCI: RoCE 10 Gb OFED stack	7.2(1)		16.04		6.3+
PCI: zEDC Java(4)	7.2(1)	12 SP1	16.04		6.3+
PCI: zEDC zlib preload library	(1)(2)	(2)	16.04		6.3+
-
-
-

(1) fixes coming, maybe in next fix pack
(2) RYO lib from https://github.com/ibm-genwqe/genwqe-user
(3) z/VM APAR VM65733 required
(4) Java 7 SR9FP30, Java 7.1 SR3FP40, Java8 SR3

(updated 9/30/2016)

How to submit requirements for LinuxONE, Linux on z Systems, z/VM and KVM on z Systems

KVM as well as LinuxONE is now part of the " Request for Enhancement (RFE) Community". So as described in my earlier post you can now also submit requirements for LinuxONE and KVM to IBM. To do this open the  community start page and then select  the "Submit" tab.
After entering in your IBM ID, you can fill in your requirement. In the product pull down please select one of:

• Linux on System z
• IBM LinuxONE
• KVM for IBM z Systems
• z/VM

Fill in the other fields as good as you can and then click on submit.

Canonical is handling requirements for the Ubuntu distribution through Launchpad. Open a bug there, put requirement in the title and tag it with s390x.

Red Hat has a defined RFE process for their customers. So after logging into RHN follow the instructions to submit a request.

SUSE requirements can be submitted to their sales reps as well as using  the "feedback" button at the bottom of the SUSE Linux Enterprise Server for System z web site.

(Updated 6/28/2016)

IBM Interconnect 2016 news

This week at Interconnect in 2016 there were some good demos that I want to share. In the key note presentation from Ross Mauri "Master Hybrid Cloud, Defend Against Cyber Threats with IBM z Systems and IBM LinuxONE"

there are some interesting demonstrations: the "shopping experience" demo for LinuxONE starts at 25:15 and blockchain demo for LinuxONE starts at 51:22.

z13s

IBM announced the new version of the business class  mainframe called z13s.  This is the successor to the zBC12. You can find the detailed announcement letter here.

z13s glas model - photo Michael Storzer

From a Linux perspective the IBM tested platforms has been updated to include z13s. SUSE has published the certifications:

• SUSE certifications: SLES11 SP3, SLES12
• Red Hat certifications: RHEL6, RHEL7

Also announced are

• z13 enhancements (see announcement letter) including a new administrative mode for Linux called IBM Dynamic Partition Manager (DPM)
• KVM for IBM z Systems V1.1.1 with a planned availability of 3/18/2016, see my separate post.
• z/VM PTFs for exploitation APARs  (see announcement letter) providing e.g. also SIMD support for guest exploitation (VM65733)
• z/VM 6.4 preview announcement

Useful links:

• IBM Redbooks:
• IBM z13 and z13s Technical Introduction, SG24-8250-01 
• IBM z13 Technical Guide, SG24-8251-01
• IBM z13s Technical Guide, SG24-8294-01 
• z13s FAQ
• 3D demonstration of the system (you can interact, open doors, get explanations etc)
• Large Systems Performance Reference for IBM z Systems updated with z13s data
• Common Criteria Certification at EAL5+ level for LPAR

(updated 10/21/2016)

SLES 12 toolchain module available for Linux on z

SUSE has released the toolchain module for Linux on System z. This is the first officially supported gcc compiler that supports the z13.
To install you need to add the product and update repository with "yast2 repositories" and then you can install it with

# zypper install sle-module-toolchain-release
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following 10 NEW packages are going to be installed:
  cpp5 gcc5 gcc5-c++ gcc5-fortran gcc5-locale libgfortran3 libstdc++6-devel-gcc5 patterns-toolchain-gcc5
  sle-module-toolchain-release sle-module-toolchain-release-POOL

The following NEW pattern is going to be installed:
  gcc5

The following NEW product is going to be installed:
  "Toolchain Module"

The following 6 recommended packages were automatically selected:
  cpp5 gcc5-c++ gcc5-fortran gcc5-locale libstdc++6-devel-gcc5 patterns-toolchain-gcc5

10 new packages to install.
Overall download size: 26.2 MiB. Already cached: 0 B  After the operation, additional 136.1 MiB will be
used.
Continue? [y/n/? shows all options] (y): y
.....


As you see you get C, C++ and Fortran. To enable z13 instructions use the -march=z13 option.

SUSE Linux Enterprise 12 Service Pack 1 (SLES 12 SP1) released

On December 22nd just in time for Christmas SUSE has released the latest updated to their flag ship server distribution.

The kernel level for 12.1 is kernel-3.12.49-11.1. As one of the top features this distribution is the first one, which allows full exploitation of the z13. And this distribution is so far the only one that's supported by IBM's version of KVM as well as Docker. SUSE has published a nice summary of the the z specific news.
And of course it has many bug fixes. So after the installation be sure to run an update to install the latest fixes from the maintweb. 

Here is my usual summary of links to more information:

• Complete SLES 12 documentation
• SLES 12.1 release notes (s390x version)
• z Systems / s390x specific information 
• release notes for High Availability Extensions (z Systems / s390x version)  
• release notes for SDK (z Systems / s390x version)
• SUSE tuning guide updated to SP1 
• Whitepaper on SUSE's module concept
• IBM documentation on developerworks
• KVM support statement
• Docker support statement

A frequently asked question is about the location of the older libstdc++ libraries that are needed for compatibility with older software products. There are two locations:

• the oldest ones are in a package called compat, that you can install with zypper directly. It has libstdc++-3-libc6.2-2-2.10.0.so and libstdc++-libc6.2-2.so.3
• the libstdc++33 is part of the Legacy Module that SUSE provides. Add it to your repositories with yast2 repositories and then you can install. It contains the libstc++.so.5

(updated 1/20/2016)

KVM for IBM z Systems

KVM 1.1.1 (new version) blog post

After the announcement in August, KVM for IBM z Systems is now generally available.  This takes the open source approach for virtualization to the mainframe. 

Here are a few links for further details:

• Announcement letter
• Supported in LPARs on zEC12, zBC12, z13 and newer machines (see announcement letter)
• Documention at the IBM Knowledgecenter and on Developerworks (pdf) 
• Redbook "Getting Started with KVM for IBM z Systems"
• LSPR numbers for KVM
• SUSE blog entry for the GA
• SUSE support statement for the upcomin SLES 12.1 in another blog entry: "With the next service pack release of SUSE Linux Enterprise Server for System z, full support for KVM for IBM z Systems will be provided."
• KVM for IBM z Systems presentation from LinuxCon
• Jens Freimann: Pushing the limits: 1000 guests per host and beyond and the presentation on YouTube
• Software support (expect this to grow over time)
• SAP application server
• IBM Software Product Compatibility Reports - select KVM for z as the hipervisor

If you encounter any problems, please open a PMR with IBM. 

(updated 12/07/2015) 

The Virtualization Cookbook for IBM z Systems - updated

The Redbook "The Virtualization Cookbook for z/VM 6.3, RHEL 6.4 and SLES 11 SP3" and the corresponding code examples got an update to the latest distribution levels. They have also reorganized the material into three different books, which I personally don't like that much - but others say this is the way to go.

• Volume 1 - z/VM 6.3 -- pdf
• Volume 2 - RHEL 7.1 -- pdf
• Volume 3 - SLES 12 -- pdf 
• code examples as tgz 

Linux on System z - Financial Trading demo from LinuxCon

Donna Dillenberger is showing a pure open source demo on the mainframe combining all the new technologies like Spark, MongoDB, Postgres, MariaDB, Node.js, Docker, Chef etc.
It's using live analytics for streams from Twitter and S&P financial data, showing the power of realtime analysis on the mainframe.
She is also providing a look at the infrastrucure driving this and even shows a fail over from one site to another using Hyperswap.
Towards the end she is providing real performance data for some of the components used.
During this presentation your probably want to pause and playback to get all the information she packed into those 18 minutes. Here is a breakdown (minutes:seconds) for reference:

• 0:00: demonstration
• 7:48: a look at the infrastructure
• 11:22: failover by killing one storage subsystem
• 13:45: performance data
• 17:00: agility & summary

(updated 8/19/2015)

SUSE Linux Enterprise Server 11 Service Pack 4 (SLES 11 SP4 ) released

On July 16th 2015 SUSE has announced the availability of the latest service pack to their SLES11 distribution. This release is a maintenance release, keeping the major kernel version stable. Here is the usual collection of links:

• release notes (s390x version) 
• z Systems spcific information for SLES 11 SP4 (11.4)
• release notes for High Availability Extensions (System z / s390x version) 
• release notes for SDK (System z / s390x version)
• kernel level: 3.0.101-63.1
• Here are some interesting features with links to the respective release notes entry:
• QSAM Access Method for Data sharing with z/OS
  This is the first stage - allowing read only access to offline devices
• 10GbE RoCE Express support
• Crypto enhancements:
• Support of Enterprise PKCS#11 (EP11)
• libica 2.4.2 available
• src_vipa: IPv6 Enablement 
• Keywords for ipl and console device for use in cio_ignore (ipldev, condev)
• Documentation
• SUSE page updated to SP4 level
• documentation as large zip file  (updated to SP4 level)
• IBM documentation for SP3 on developerworks
• SUSE tuning guide updated to SP4

SUSE Linux Enterprise Server 12 (SLES 12) released

In November last year SUSE has announced the availability of their next major distribution release SUSE Linux Enterprise Server (SLES 12). It's an interesting change in strategy that SUSE chose to release it after the competing RHEL 7 distribution. In the past SUSE has always been first to the market.

Here is a the usual list of interesting information links:

• release notes (s390x version) 
• release notes for High Availability Extensions (System z / s390x version) 
• release notes for SDK (System z / s390x version)
• kernel level for GA: 3.12.28-4.6
• this new distribution will only run on z196 and later systems:
• IBM tested platforms
• SUSE certifications: z196, z114, zEC12, zBC12, z13
• Documentation
• SUSE documentation
• IBM documentation on developerworks
• SUSE tuning guide
• SUSE® Linux Enterprise Server 12 for System z presentation at GSE
• SUSE Linux Enterprise Server for System z Current & Future Features presentation at SHARE

SUSE has released the first maintenance web kernel for SLES12  (kernel-default-3.12.36-38.1.s390x.rpm)
It contains quite some fixes that you really want to have on your system.

z13

Problem with the latest crypto device driver updates on RHEL 6.6 / SLES 11.3 on older hardware

With the introduction of the code for more crypto domains (required for z13) into the service streams of RHEL 6.6 (kernel 2.6.32-504-3.3 and 2.6.32-504.8.1), SLES 11.3 (kernel 3.0.101-42-1) and SLES12 (kernel 3.12.32-33.1) an unfortunate regression has been introduced. The crypto card support for z196 / z114 and older machines is no longer working with the new code. If you are in that situation, please update to the levels below.

This has been fixed in the following kernel levels.

• SLES12 (kernel 3.12.36-38.1 and later)
• SLES11 SP3 (kernel 3.0.101-0.47.50.1 and later)
• RHEL 6 (kernel-2.6.32-504.16.2 and later)

(updated 4/23/2015)

Linux performance guides for distributions

I'm often asked for the performance guides from Red Hat and SUSE. So here are the links for that:

• RHEL 6 Performance Tuning Guide 
• RHEL 7 Performance Tuning Guide
• SLES 11 SP3 System Analysis and Tuning Guide

(updated 7/72014)

IBM XL C/C++ for Linux on System z beta program

IBM has launched a beta program for a new compiler for Linux on System z. If you've been working with IBM platforms you may already know the xlc compiler. This compiler is now also available as a beta for Linux on System z as well.
Interested customers and software vendors should register for it at this registration link (free IBM ID required).

The key difference to the gcc included in the distributions is that this compiler is independent of the distribution. So as a software developer you can use this compiler and more important the generated binaries on all the supported Linux distribution. By that you can get improvements like zEC12 exploitation earlier than waiting for the next gcc in a distribution.

And my personal hope is that the performance will also be good as well. We will know this when the compiler is generally available.

Oprofile on zLinux - how to setup and use

(updated 1/8/2014)

Oprofile is a system wide profiler which is available in all major distributions for Linux on System z. Here is the setup and use for RHEL 6 and SLES 11.

The following example is for RHEL 6.5 (kernel 2.6.32-431.1.2.el6.s390x). For other kernel levels you need to adapt the package numbers. But basically it should work the same. The Red Hat description is here.

First step: install the required packages:

• oprofile-0.9.7-1.el6.s390x
• oprofile-jit-0.9.7-1.s390x (only needed for profiling Java code)
• oprofile-gui-0.9.7-1.el6.s390x (only needed if you want the GUI)
• kernel-debuginfo-2.6.32-431.1.2.el6.s390x.rpm
• kernel-debuginfo-common-s390x-2.6.32-431.1.2.el6.s390x.rpm

Note that the kernel-debuginfo packages are only available on RHN. See this howto get it. You need to log into your Red Hat Customer portal for the full information. Also install from RHN any other debuginfo package of a distribution package you want to analyze. 

Second step: configure oprofile

opcontrol --setup --vmlinux=/usr/lib/debug/lib/modules/`uname -r`/vmlinux

Third step: measure workload

opcontrol --start

run your workload

opcontrol --stop

opcontrol --dump 

Last step: call opreport or opannotate with the options you want. For understanding options use the respective man pages. One commonly used option is:

opreport --symbols

Don't be surprised by an entry with symbol name vtime_stop_cpu. That's cpu  idle in RHEL 6.

For SLES 11 SP3 the setup is similar. SUSE has a good description on how to use in their Systems Analysis and Tuning Guide.

So in the first step you need to install oprofile-0.9.8-0.13.31.s390x.rpm from the SDK. Optionally  the kernel debuginfo package e.g. kernel-default-debuginfo-3.0.76-0.11.1.s390x.rpm as well as all the debuginfo versions of distribution packages you want to profile.

The vmlinux file for SLES is gzipped in /boot. If you have enough space there you can just gunzip it in place otherwise put it in /tmp as the SUSE guide suggests. Then in the second step you set up oprofile by

opcontrol --setup --vmlinux=/boot/vmlinux-`uname -r` 

Step 3 & 4 for SLES 11 are the same as above. 

If you want to analyze data on another system use oparchive. It will generate a directory with all required data that you can compress and take off the system. So e.g.

oparchive -p <path to Linux modules> -o /tmp/myoutputdir

You can also include Java and JITed code into the profiling by adding

-agentlib:jvmti_oprofile 

to your Java options. For SLES11 you need to add /usr/lib64/oprofile to your LD_LIBRARY_PATH. This especially valuable if you don't know yet where to search for a problem. If you have identified Java code as the problem then a specialized profiler is probably the better choice.