ClickCease Live Patching In the DevOps Workflow - TuxCare

Join Our Popular Newsletter

Join 4,500+ Linux & Open Source Professionals!

2x a month. No spam.

Live Patching In the DevOps Workflow

by DeShea Witcher

December 2, 2020 - VP of Marketing

DevOps is a combination of software development and IT operations that aims to improve and evolve products at a faster than normal pace in order to help organizations compete more effectively and better serve their customers.

The DevOps workflow is characterized by quick release cycles that require quickly tested code. This is done by giving developers the ability to quickly spin up new virtual machines, usually based on templates, where they can run those tests. This is controlled with scripts, using at least one of multiple automation tools available, such as ansible or puppet.

Developers will typically spin up new virtual machines to test, build, and release new code but are rarely aware that those virtual machines can be targeted by malicious actors, even if they are short-lived. Having such a system compromised can provide a stepping-stone for access to internal assets that would otherwise be unreachable. This has been a known fact going back to at least 2015, when the topic was discussed at a Black Hat Europe 2015, with the appropriate title of “Continuous Intrusion: Why CI tools are an attacker’s best friends,” and yet the risk is still downplayed and ignored.

With this article, you can learn how to integrate KernelCare deployment into the workflow so that even those systems are kept up-to-date with the latest kernel patches.

DevOps Virtual Machines

DevOps uses virtualization to help them have rapid and stable deployments. “Virtualization is the process of creating software that mimics various hardware and software environments without having to change out the physical hardware itself” (BMC). A virtual machine is intended to function as a simulation of a specific hardware or software configuration, which allows DevOps to test different aspects of software or hardware without having to change out any physical components; they can also run multiple simulations at once, speeding up the process.

Security Concerns

The virtual machines are based on templates created at some point in time, and when they are brought online, there is usually no updating process, so whatever vulnerability existed at the time of creation or whatever vulnerabilities appeared since that creation will be in the virtual machine that is brought online. This risk is downplayed by the usual short lifetime of these virtual machines.

“You can take a snapshot of a virtual machine and write it off to disk, so you don’t have to recreate it the next time, or for disaster recovery. Just fire off one of these virtual machines sitting in offline libraries. But for the most part, they’re not being kept up to date with A/V signatures and patches,” said Neil MacDonald.

Those virtual machines are usually “throwaway” virtual machines. They are spun up, used for testing, and either deleted or forgotten. The problem is that these virtual machines are still technically servers, and if they are not patched and maintained, the vulnerabilities that existed when it was made are still there, meaning they are ripe targets for hackers. While they should be checking for patches or vulnerabilities, they often do not, so these templates are a liability waiting to happen.

Hackers have identified this possible hole in security and can lay in wait for the virtual machines to appear and attack them during that time. These can be just the entry point into the rest of the infrastructure because these virtual machines require access to databases, other internal systems, or authentication mechanisms.

What is DevSecOps?

The simple definition of DevSecOps is that it adds security to the development and operations. It adds the objective to implement security into the DevOps pipeline to ensure that everything is kept safe and secure. Shifting an organization toward following a DevSecOps framework can be difficult since it often requires extra work monitoring the information in the virtual machines. However, KernelCare offers live patching so that you do not have to do any extra work to help make systems safer and more secure. Our system will take care of everything for you regarding kernel and shared libraries security patches.

KernelCare DevSecOps Solution

Since security and live patching are important to the DevOps process, a simple solution is needed: KernelCare. KernelCare has a simple installation procedure that can be automated and included in the deployment scripts for the virtual machines. This way, when they are brought online, KernelCare will live patch the kernel and/or shared libraries and assist in making them more secure, however, brief their lifetime may be. This also ensures that any snapshots you make of virtual machines will be patched when they are brought online, so you do not need to worry about old vulnerabilities leaving doors open for hackers to get into your systems.

If the business uses containers instead of virtual machines, then installing KernelCare on the container host is still recommended. That way, you can have all your containers protected without needing to reboot the host at any point to apply security patches and no downtime.

In the DevOps pipeline, the integration described here happens between the build and the test phases.

Integrating KernelCare Into Your Systems

To integrate KernelCare with Ansible, you need to give the following information:

  •         “ePortal server name (or IP) in the eportal srv Ansible variable. Other config file options can be found at Config Options and KernelCare client config file (ePortal).
  •         “An activation key in the activation_key Ansible variable. Activation keys can be generated in ePortal as described in Managing Keys (ePortal).

To integrate KernelCare with Puppet, follow the instructions in the linked video.

A code example of integrating KernelCare into your system is:

– hosts: kernelcare

  vars:

       eportal_srv: http://192.168.250.19

       activation_key: 89gRVCp1rY0ZQ053

tasks:

– name:Download the installation shell script

       get_url:

       url: “{{ eportal_srv }}/installer”

       dest: /root/kc-install.sh

       mode: ‘0700’

– name:Run the installation shell script

       shell: /root/kc-install.sh >> /var/log/kcare_install.log

       environment:

       KCARE_REPO: “{{ eportal_srv }}/repo”

– name:Update kcare.conf with ePortal configuration

       blockinfile:

       path: /etc/sysconfig/kcare/kcare.conf

       create: yes

       block: |

       PATCH_SERVER={{ eportal_srv }}/

       REGISTRATION_URL={{ eportal_srv }}/admin/api/kcare

– name:register KernelCare agents

       command: /usr/bin/kcarectl –register {{ activation_key }}

Conclusion

With KernelCare, any patches will be installed during your DevOps pipeline’s deployment phase, with no extra effort for the developers when creating virtual machines. You will have a more secure environment for your testing. This will, in turn, lead to a more secure internal environment. KernelCare is a step in the right direction, assisting in upgrading your DevOps to DevSecOps. KernelCare live patching can easily integrate into the deployment scripts for new virtual machines and thus make them more hardened against vulnerabilities; with KernelCare, you get more security without extra work.

Looking to automate vulnerability patching without kernel reboots, system downtime, or scheduled maintenance windows?

Become a TuxCare Guest Writer

Mail

Help Us Understand
the Linux Landscape!

Complete our survey on the state of Open Source and you could win one of several prizes, with the top prize valued at $500!

Your expertise is needed to shape the future of Enterprise Linux!