October 5, 2012

Large pages in Linux on System z - howto enable in Linux

Since the z10 in 2008 the System z hardware offers large pages. The size is 1M and the main purpose is the same as on any other platform: reduce the memory footprint in the operating system for page tables and reduce the TLB misses in the hardware.
Linux on System z offers support for large pages. Right now they are not pageable, so only the required number should be allocated. There are two ways to enable it:
  1. Add hugepages=<npages> to the kernel parameter line, where npages is the number of 1M pages you want to allocate. So e.g. hugepages=1024 will allocate 1GB worth of memory as large pages.
  2. Dynamically: do a "echo <npages> > /proc/sys/vm/nr_hugepages" to receive npages worth of large pages. This only works if enough free memory is available that can be remapped as large pages. For 1GB of large pages "echo 1024 > /proc/sys/vm/nr_hugepages" will try to allocate them.
To check if the allocation worked, do a "cat /proc/sys/vm/nr_hugepages". Usage information can be as usual found in /proc/meminfo.

Note that if you are running in an environment, which does not support large pages, you can still enable them and get the memory savings as Linux will emulate large pages for the applications.

Finally if you are an application programmer and want to make use of large pages take a look at Documentation/vm/hugetlbpage.txt in the kernel source tree. There you find the supported interfaces for your kernel level.