Discovering Ksplice – Oracle Linux 8 – Patching Doesn’t Have to Mean Rebooting

Share this post on:

Discovering Ksplice

Ksplice is easy to configure. There are a few concepts and points you need to understand before Ksplicing your servers:

  • If your servers are running on Oracle Cloud Infrastructure (OCI), using Oracle’s default platform images, all the Ksplice preparation work and access to the Ksplice repo is already performed to support online Ksplice usage. Systems will also need a route (usually done with a NAT gateway) to the Ksplice endpoint. This is automatically set up when using the Virtual Cloud Network (VCN) wizard.
  • If you are using Autonomous Linux on OCI, your systems are already being Kspliced automatically.
  • There are two Ksplice clients: the legacy Uptrack client and the Enhanced client.
  • You do not need internet access to use Ksplice; you can use specially built Ksplice .rpm files using the offline mode. This will require that you mirror the ULN Ksplice repos on a local YUM server. This was covered in Chapter 5.

Both the Enhanced client and the Uptrack client allow you to Ksplice the kernel, but the Enhanced client also offers the ability to Ksplice user space libraries, KVM hypervisors, and Arm-based systems and offers Known Exploit Detection on x86_64 platforms. Most new users should use the Enhanced client.

The older Uptrack client does not offer the advantages the Enhanced client offers, but it does support the ability to patch some non-Oracle Linux systems.

Ksplice offers two different modes: the online mode and the offline mode. The online mode requires that each system registers to the Oracle ULN but offers a Ksplice web interface, the Ksplice Uptrack API, for advanced automation and the ability to patch a Xen hypervisor. However, since this requires every system to not only register with ULN but also for internet access, many system administrators use the offline mode.

The offline mode allows the system to use Ksplice without internet access. This is most often used by Enterprise and Government environments where all servers do not have internet access. In these use cases, the most common method is to set up a ULN mirror, giving that system access to the internet, and the local Oracle Linux servers will pull from that mirror to use Ksplice:

Figure 7.5 – Ksplice offline mode using RPMs

With the offline mode, only the local ULN mirror needs to be able to access the Oracle ULN, via the HTTPS protocol. This can be done via a basic firewall rule or a more advanced proxy server. Once the ULN mirror is registered and synced from the ULN, the servers behind the firewall can pull their RPMs via HTTP/HTTPS from the ULN mirror. There are a couple of advantages to the offline mode: less bandwidth is used for the internet and not every client needs to be registered. Of course, having no API, there are some limitations with more advanced automation.

Regardless of what mode is used, the clients must have access to a few specific Ksplice channels. The following table describes the channels that are available for Ksplice in Oracle Linux:

Table 7.1 – Ksplice channels

Note

If prelink is installed, revert the pre-linked binaries to their original state (prelink -au) and then uninstall prelink (dnf remove prelink -y). While not often used, prelink is not compatible with Ksplice.

Now that you have an understanding of why you need to use Ksplice and the basics of how it works, let’s get to the recipe.

The easiest way to use Ksplice is to allow each system to access the Oracle Ksplice service directly. This is called online mode. This can be done via an internet proxy, direct access, or by running the systems in OCI.

Getting ready

To do this, you will need a test system, running Oracle Linux 8, with access to the internet.

How to do it…

When using Ksplice, make sure you match each system for both the operating system version, and the user-space libraries. Do not use an Oracle Linux 9 repo for an Oracle Linux 8 system. An example of what a system should look like can be seen in the following screenshot:

Figure 7.6 – Required channels

If you are using an internet proxy server, you will need to export some environmental variables to set the proxy servers:
sudo http_proxy=http://proxy_server_URL:http_port
sudo https_proxy=http://proxy_server_URL:https_port
sudo export http_proxy https_proxy

Set proxy_server_URL to the proxy server, and http_port to the port the proxy server uses.

Next, Ksplice will be installed using dnf. Both the ksplice and uptrack packages are needed:
dnf install -y ksplice uptrack

When the installation is complete, please verify that your system is seen in the ULN and that an access key has been populated in /etc/uptrack/uptrack.conf.

The access key should be in the [Auth] section and should look similar to this:
[Auth]
accesskey = b3ag33k1746d141edb377f6f643344e23ad9638ae4d896ead4dcdddbb350a058b

Once done, perform a normal dnf update and then reboot the system. This dnf update is required so the Ksplice-aware user space libraries are loaded; it is only required the first time Ksplice is used.

Once the server is rebooted, Ksplice can now be used to patch user space libraries and the kernel.

To see what updates are available for the kernel, you can run the Ksplice kernel upgrade command, passing -n for no action:


ksplice -n kernel upgrade

The output from this command is seen in the following screenshot:

Figure 7.7 – Ksplice available upgrades

In this system, only one update is available, patching CVE-2021-4034. To splice the system, run the same command, replacing -n with -y to say yes to applying the kernel path:

Figure 7.8 – Ksplice updating the kernel

From here, you can see that the CVE patch has now taken place without a reboot!

The kernel that has been patched is referred to as the effective kernel. When you use the ksplice kernel uname -r command in Ksplice, it displays the effective kernel version, which indicates the current security status of the kernel based on the applied patches. This version typically varies from the initially booted kernel version and is meant to reflect the present condition of the kernel regarding any potential security threats or major issues.

You can now validate what your effective kernel is by using the following Ksplice command:
ksplice kernel show

Note that the version of the running kernel and the installed kernel will not match! This is because when Ksplice splices the kernel, it splices the RAM of the running kernel with the new code. This patches the kernel (and selected user space libraries) but has the effect of uname no longer matching the effective kernel:

Figure 7.9 – Ksplice uname

Here, the installed kernel is 5.4.17-2136.302.7.2.1.el8uek.aarch64, but the effective kernel that has been Kspliced is a newer kernel at version 5.4.17-2136.306.1.3.el8uek. This is normal behavior with Ksplice.

User space libraries can be patched in the same way, replacing the kernel with the user. As a note, if the Ksplice user space libraries are not installed, the first time the command is run, you will be prompted to upgrade the user libraries to a Ksplice-compatible version:

Figure 7.10 – Installing Ksplice user libraries for the first time

Share this post on:

Author: Stacy Atkins Prince

View all posts by Stacy Atkins Prince >

Leave a Reply

Your email address will not be published. Required fields are marked *