There is no doubt the acceptance and growing popularity of Linux on Power is here. With POWER-based systems providing advantages over x86 systems in performance, capacity, memory and IO bandwidth, as well as reliability, and with the latest Linux distributions from RedHat, S– USE, and Ubuntu exploiting POWER8 technology, businesses are now looking at hosting mission-critical applications on Power Systems with Linux for a better customer experience at a lower TCO. And, Linux is Linux, right??….by the way, not really! But that will be the subject of a different BLOG. (I have intentionally left out the CAPI technology from this discussion as I am saving that topic for another BLOG along with the incredible capabilities it provides.)
Yes, Linux on Power has great capabilities and will expand rapidly, but I also believe you have several critical decisions to understand, as you start your implementations. Since everyone is not actively running production on Linux on Power, I suspect that most all of you will start with a Proof of Concept or a Proof of Technology. And, this is where decisions have to be made and options chosen. In my opinion, your decisions have to start at the application layer and then work down to the hardware server selection. Several years ago the choices were much easier since there were only specific elements available such as the virtualization layer (it was only PowerVM), the Linux distribution (previously only S– USE and RedHat were certified), the Cloud (Smart Cloud Entry only), Endianness (Big only), and the hardware support of POWER6 or POWER7 really only differentiated by the SMT levels. This has now changed with many new options and the IBM support of key Open Sources technologies like OpenStack and KVM. And, like Indiana Jones was told by the Grail Knight in the Last Crusade, “choose wisely”. You should also choose wisely, and with forethought.
A suggestion of mine is to start at the application. First, the application must be available (i.e, compiled) for POWER or must be a script or scripting tool. Or, it must have a Java application that is ISA neutral or can be compiled to the POWER ISA (PPC) such as Open Source code or your own “home grown” application. Choose whatever application you will (i.e Websphere, SAP HANA, TSM, MariaDB, Redis, home grown, etc.) for the POC or POT, but you then must determine the Endianness of the application or the Endianness that you will compile for. The reason is that the Endianness of the application MUST match that of the Linux Distribution. So, if you choose SAP, which is Big Endian, you need to choose a BE distribution like RedHat 6.5, RedHat 7.1 BE (not the LE RedHat 7.1 which is a separate SKU), or SLES 11. If you had chosen SLES 12 or Ubuntu, you would have a problem since these are Little Endian distributions, and your SAP HANA test would fail immediately. Now, skipping forward several choices, we must recognize that Little Endian applications can only run on POWER8 hardware. It can run in a partition, IFL on a POWER8 server (S8xx, E8xx) or on an S8xxL system, but not on POWER7 nor POWER7+ nor POWER6.
Many people ask, “What distribution should I use?” First, you must choose the correct distribution, based on LE or BE capabilities. Then, the differentiator is normally based on consistency. RedHat, S– USE, Ubuntu, Debian, Fedora, OpenSuse are all based on the Linux kernel for sure. But, they are all at different kernel levels, have different “support packages” included in the distribution?, and have different administrative procedures for such items as tuning, install, management, EUI, support, etc. You should review specific requirements that may lead to a multi distribution environment, versus consistency within the environment, for a single skill set and single set of procedures. It should be noted that the interface to the Linux OS is consistent for applications, so that is not a major consideration, although certain ISV applications may only be certified on specific distributions and releases. An example of that is SAP HANA, which is only supported on SLES 11, at this time.
Another point, though not that significant, is that the different distributions have different prices and T&Cs, such as pricing by cores or sockets and/or virtual machines. These should be reviewed and ordered correctly for entitlement compliance.
Now choose the support structure for Linux on Power. You have two choices basically… support from the distributor or support from IBM GTS. Each are good choices, as both IBM and the distributor interact for fix resolution, but there are differences in costs… ie, IBM charges for support for the entire server at a single price and distributors charge based on the entitlements. Another point is that if Ubuntu support is wanted, it can only be provided by IBM since Ubuntu is normally community support. Again, consistency within the Enterprise should lead to a clear decision here.
Now you need to decide on your virtualization technology. Your choices are PowerVM, KVM, or no virtualization, sometimes called bare metal. Bare metal implies no virtualization or associated overhead, and as such can provide the highest performance of Linux applications on POWER8 Systems. If this is what you wish, for example, with analytics, HPC, etc, you can only run Linux on Power on POWER8 Scale Out systems (S8xx L’s) since only they have a hardware interface layer called OPAL (Open Power Abstraction Layer) that the Linux distributions must be enabled for, in order to operate. OPAL, as of today, is only available on the S8xx L systems and not the Enterprise systems (E8xx) nor POWER7 systems. Although it will change soon, only Ubuntu (LE only) and SLES 12 (LE only) are enabled for bare metal (direct interface to OPAL). PowerVM is the tried and true high-function virtualization technology for Power Systems. As of the 2.3.3.50 release, PowerVM will support concurrent usage of BE and LE Linux distributions, with equally supported functionality. However, PowerVM is specific to Power Systems, and some customers may wish a consistent virtualization technology between Power and x86 systems. If this is the case, then KVM, specifically the PowerKVM product from IBM, should be considered. PowerKVM needs the OPAL interface to access the hardware, and as such, will only run on the POWER8 Scale out Linux-only servers at this time. So, no running PowerKVM on POWER7 systems nor on Enterprise Power Systems. But, if chosen to be the virtualization layer on the POWER8 Scale Out Linux-only servers, PowerKVM will support BE and LE distributions and applications. Since most of the internals are rooted in KVM, consistency with x86 procedures can be implemented both administratively and skill wise. However, if you are going to use RedHat distributions of Linux and RedHat as your Linux support provider, realize that RedHat support does not recognize PowerKVM as a supported environment, and you should obtain RedHat Smart Virtualization as the supported by RedHat virtualization Layer.
So far, I have a targeted application, Linux distribution, support, and a virtualization layer. Want Cloud enablement for automated provisioning? The Open Source community is working on OpenStack and IBM has embraced this in PowerVC, which is OpenStack productized for Power architecture. PowerVC has replaced the predecessor product VMControl, which ran with System Director. OpenStack provides the ability to capture, deploy, and destroy VMs in the Power environment. You can run with PowerVC as your highest level, or you can run with all the self-provisioning capabilities provided by IBM Cloud Manager with OpenStack or IBM Cloud Orchestrator. ICO provides broader platform support and additional business oriented functions. By the way, some people ask about VMware. The answer is no. VMware is only supported running on x86 systems.
Choosing the cloud layer and the virtualization layer also involves some planning. Since ICMOS contains OpenStack, you cannot run both PowerVC and ICMOS on PowerKVM. The reason being is you cannot have two OpenStacks managing one virtualization layer. But, if you choose to run PowerVM, both ICMOS and PowerVC talk to the APIs of the HMC (which must be at the 8.x.x level) to provide the automated virtualization. Again, plan ahead.
Finally, you need to select a hardware platform. POWER7 is perfectly valid, but you must recognize that if POWER7 is your choice, you cannot run LE nor KVM. With POWER7, you will get BE support from S– USE, Redhat, Fedora and Open S– USE, but no Ubuntu (again LE only). As you move to the POWER8 technology, you must realize, at this time, OPAL is only implemented on the POWER8 Scale Out systems for Linux only (S8xx-L models). Since OPAL and pHyp are both shipped on the “L” models, you can reboot and switch back and forth to support PowerKVM or PowerVM. If you plan to do this for test environments, it may be easier to order the PowerVM version and order PowerKVM as an MES later, since PowerVM requires vet codes and PowerKVM does not.
Here is a table that hopefully will clarify the options and be aware that this is a very dynamic space with options changing frequently as certification tests complete. So, if the environment you wish is not represented here, it may be a timing issue and you should ask for the latest supported stacks:
My intent of this writing was to provide you with insight into the options and decisions you will need to be aware of when implementing a POC or POT for Linux on Power. It is kind of intricate, and there are always several people that can go deeper into the systems and infrastructure layer; but my opinion is, you run systems to run applications and not operating systems. As such, start with the application and work down, instead of choosing a server then starting to fill in the pieces. Doing so may create an environment that does not support your application as you thought it would.