Director of Power Systems
Linux on Power is a very dynamic environment.
Be aware that technology and support changes, and will be changing very quickly, normally to the better. As soon as one finally understands the characteristics of the environment and defines a plan for implementation, it may be that before you can execute your plan, capabilities may have improved to the better and limitations removed. It is recommended to always check the latest information. Take, for example, virtualization support. Several months ago, there were limitations and technology gaps…these have been updated recently, all to the better. Let’s look at virtualization of the PowerLinux environment.
Currently, there are four “virtualization” models that can be implemented on PowerLinux systems: PowerVM, PowerKVM, RHEV (RedHat Enterprise Virtualization, aka Smart Virtualization), and bare metal… although bare metal implies no virtualization. Let’s look at each of these implementations.
PowerVM supports all PowerLinux distributions as guests (aka virtual machines, aka VMs) on all Power Systems including the PowerLinux systems 812L, 822L, and the non-GPU 824L. PowerVM 2.2.3.5 supports both BE and LE distributions, with LE only enabled on POWER8 systems. PowerVM 2.2.3.5 will support AIX, IBM i, RedHat 6, 7, SLES 11, 12, Fedora, Open S– USE, and Ubuntu, all simultaneously, in different virtual machines (aka LPARS). PowerVM 2.2.3.5 will also require hardware microcode level 830 to be installed. (And, at this time with the 830.0 release, if you install 830, that will exclude support of OPAL, so you cannot switch back and forth…. but, more on that in a couple of paragraphs.) If the PowerVM virtualization environment is earlier than 2.2.3.5, then only BE Linux distribution are supported (hence, no Ubuntu nor SLES 12 nor RH 7.1 LE). Since PowerVM is the only virtualization environment on non-L servers (ignoring bare metal), decisions and ordering virtualization on the non-L models is easy since there is no choice. This ordering process also enables the pHyp virtualization layer to be initialized on the Power System. And, I should point out that all of the features of PowerVM support Linux virtual machines, including micro partitions, VIOS, NPIV, AMS, AMD, etc., etc. Existing Power customers, using PowerVM, may find that many of the features they like (NPIV, expansion drawer support, VIOS, AMS, Shared Pools, micro partitions, etc.) are not supported on PowerKVM. While I do not intend to explore every feature or function difference, between PowerVM and PowerKVM, that exercise should be undertaken if you are trying to decide between PowerVM and PowerKVM, as the virtualization engine.
A significant point is that there are two PowerVM’s. One is called PowerVM, which comes in standard and enterprise versions; and the other one is called PowerVM for PowerLinux. PowerVM for PowerLinux only supports Linux guests, and can run on any “L” model. PowerVM for PowerLinux is logically equivalent to PowerVM Enterprise Edition, and includes Live Partition Mobility.
PowerKVM is the opensource-based virtualization technology available on POWER8 systems, and only supported on the “L” models. KVM stands for Kernel Virtual Machine… not Keyboard, Video, Mouse. PowerKVM does not support the pHyp interface, so OPAL (OpenPower Abstraction Layer), which is an alternative hardware interface, was created. OPAL is only available on the POWER8 Linux-only servers S812L, S822L, and S824L. This implies that you can only run PowerKVM on those models, and NOT on the S824L GPU model, E870, E880, E850, S812, S822, S814, S824, nor any POWER7 systems. As noted above, the L models can run PowerVM, and now we know they can also run PowerKVM. However, you must run the correct hardware interface layer… either pHyp or OPAL, both of which are shipped on the “L” models systems. When you initialize the “L” models, the HMC ASMI interface gives you the option of initializing either OPAL or pHyp (this may appear as KVM or PowerVM in the interface). Choose the layer that matches your virtualization technology. One interesting item is that you can always reboot the server and switch to the other hardware interface layer if you wish. This means you can start with PowerVM, and if you feel you really want PowerKVM later, you can reboot and set the system to OPAL. If you may do this, or you may switch from PowerKVM to PowerVM, I suggest that you order the system with PowerVM. This way you will get both the vet codes and machine code for PowerVM, entitled to you, in the initial order. As PowerKVM has no special vet codes, you could order PowerKVM separately, and then to get it going, just reboot, pick OPAL, and install PowerKVM.
PowerKVM is a Power Systems enabled version of the open source KVM, and hence, is very similar to the x86 KVM technology. IBM has added code to interface to the OPAL layer, and therefore, has PowerKVM, which is slightly changed from the OpenSource code, and is supported by IBM. Operationally, x86 KVM and PowerKVM are the same… such as functionality, features, Kimchi interface, etc. (Not to do an extensive PowerKVM presentation, but a quick view of KVM is that KVM is a virtualization layer running RedHat, that then supports guests that can be RedHat, Ubuntu, or SLES; and the guests can be LE or BE…. plus PowerKVM supports processor and memory sharing; over commitment; dynamic addition & removal of virtual devices; live VM migration; and it enables higher availability and allows workload balancing; exploits P8 features like micro-threading, providing scheduling granularity; exploits performance, scalability and security that is built into Linux; and can be managed by PowerVC, and other opensource tools, which provide familiar Linux administrative capabilities. (This is not a plug for PowerKVM, but merely a quick function list for those more familiar with PowerVM.)
It should be noted, that based on RedHat’s defined supported environments, PowerKVM is NOT supported as a virtualization technique for RHEL running on Power Systems. RedHat states that RedHat Enterprise Virtualization RHEV is one of the only software virtualization environments that is a RHEL supported environment (in addition to VMware and HyperV). It is based on OpenSource KVM. PowerVM and zVM are considered hardware virtualization, and are supported environments, also under the RedHat definitions. PowerKVM works fine with RedHat distributions since it is based on OpenSource KVM, but RedHat does not provide support for RedHat Enterprise Linux, if it is running under PowerKVM. The good news is that since RHEL is a significant market leader, they have enabled RHEV on Power Systems. In today’s parlance, RHEV is also referred to as Smart Virtualization. So, consider using RHEV if you wish to use KVM as the virtualization engine, especially if you wish to use RedHat support. The benefit of using KVM is that the virtualization of x86 and Power Systems can be the same, namely RHEV, providing consistent capabilities. Both can be managed by RHEV-M (Smart Management) on an x86 system. Further on this, RedHat “support” says if you have a problem running under PowerKVM, you must reproduce the condition under PowerVM, or bare metal (not yet supported on Power), and they will provide support for that issue, in that environment. However, switching between PowerKVM and PowerVM, and having both installed and enabled for support reasons, is quite burdensome. IBM GTS has the same position, but they will do the recreation for you, and possibly provide a fix to the issue. I mentioned that this is a changing environment, and I hope this is one thing that changes in the future.
Bare Metal is the term used to designate that no virtualization layer is implemented between the Linux OS and the hardware. This implies that no “overhead” of the virtualization software is present and maximum performance is the result. Sometimes, the virtualization overhead can be from 3% to 20%, depending on many variables such as IO queueing and sharing, and context switching. Some applications, like HPC and high frequency interactions, cannot tolerate any overhead. This is where running the Linux OS directly on the hardware provides maximum performance. Right now, only Ubuntu is enabled for bare metal, and since this does require OPAL, this implies that bare metal is only on the “L” models. It may be that “near” bare metal can be achieved in the PowerVM environment by using dedicated-donating partitions. Using dedicated cores and dedicated IO can provide optimal performance in a virtualized environment.
So, that is a quick summary of the virtualization options available for Linux when running on Power Systems.
I know many people like VMware, but VMware does NOT run on Power Systems, so that option is NOT available.