New Kernel Vulnerability Found by Virtuozzo Live-Patched by KernelCare
A month ago, Virtuozzo‘s Team discovered the new security vulnerability in the kernel – CVE-2020-14305. It corrupts the memory in kernels from v3.5 to v4.10 and affects various Linux distributions. KernelCare is preparing the patches for this CVE which will be available this week. Read this article to learn about how the vulnerability was discovered and what is the mitigation for it.
A memory corruption CVE (CVE-2020-14305) was discovered at the beginning of June by Linux kernel engineer of Virtuozzo, Vasily Averin. The vulnerability affects various Linux distributions: Red Hat Enterprise Linux 7, Debian 8 & 9, Ubuntu 14.04 and 16.04, UEK 3 & 4, Centos 7 ALT and kernels from 3.5 to 4.10. If IPV6 port 1720 is open on the node, connect to it corrupts the memory. Particularly, FastVPS team – the user that reported this bug to Virtuozzo team – has experienced the corruption of an element in the kmalloc-192 slab. The vulnerability is fixed in Virtuozzo Hybrid Server 7.0 Update14 and ReadyKernel 108.
According to RedHat, This issue is rated as having Moderate impact because of being limited to only IPV6 port 1720 being used and if a particular module for Voice Over IP H.323.
To address this issue, the Virtuozzo team has created a patch for Virtuozzo 7 and KernelCare team has created the patches for other affected Linux distros for rebootless distribution.
KernelCare team wants to thank Virtuozzo team and Vasily Averin for finding this bug and giving us a chance to provide our Clients with the patches against CVE-2020-14305 so quickly.
KernelCare Patches Release Schedule
Here is a schedule of KernelCare patches release:
- Monday, July 6: RHEL7, Debian8 & 9, Ubuntu 14.04 & 16.04,
- Thursday, July 9: Unbreakable Enterprise Kernel 3 & 4, CentOS 7.
No special mitigation needed for patching of this vulnerability. If your kernel is affected and you are using KernelCare – the patches will be applied automatically based on your update settings (by default, KernelCare agent checks for the patches every 4 hours).
If you are using your KernelCare – your patches will be delivered automatically by KernelCare and you don’t need to do anything extra. If not – this is the right time to sign up for a 30-day trial. No credit card required.
How CVE-2020-14305 was discovered?
The Virtuozzo Team has provided KernelCare with the exclusive details on how this CVE was discovered. It has become possible with the help of their client – FastVPS team.
“I want to thank the FastVPS Team for reporting this bug, their commitment, patience and collaboration in this regard. Without their collaboration, this vulnerability might not be discovered at all”, Vasily Averin, Linux kernel engineer at Virtuozzo, said.
Continue reading to learn the whole story.
Searching through crash dumps Virtuozzo Team found out a cause of crashes – it was the memory corruption in the kmalloc-192 slab where different kernel objects from 128 to 192 Bytes are stored. They already received reports about such objects corruption earlier but each time investigations hit the wall.
It was very difficult to investigate memory corruption by virtue of its nature. Often memory corruptions simply elude detection, for example, if unused memory corrupted. Sometimes you can have luck when corruption doesn’t harm something critical. But even if it leads to a kernel crash it is hard to understand that the crash reason was memory corruption.
It is even harder to understand the nature of corruption. Usually, this is due to use-after-free and therefore you have to check allocation, initialization, reference counting, and releasing corrupted objects. However, in the case of kmalloc-192 slab, things are worse–it is used to keep a lot of different objects. Virtuozzo Team has tried to track the most popular and failed.
However, from time to time the Virtuozzo team found something. In February, Vasily discovered a bug that caused memory corruption in kmalloc-192.
He hoped that he has resolved the issue. But one day FastVPS came with the same problem. Vasily’s Team asked them to reproduce the issue on the kernel with the above fix but this didn’t help, the issue repeated.
Usually, in such cases, Virtuozzo uses the kernel debug with the additional all-purpose debug. Customers don’t use debug kernels in production as it dramatically decreases performance and it is totally unacceptable for our hosts. However, FastVPS moved the debug kernel to production. They suffered some time, but since the problem didn’t reproduce, they switched back to the regular kernel.
To catch the issue on the regular kernel, Virtuozzo asked FastVPN to enable the slub_debug.
It also decreases performance and increases memory consumption, makes additional memory checks on allocating and releasing objects, and allows you to track how a corrupted object was used in the past. But still, the issue didn’t reproduce and nodes with enabled slub_debug ran well. But in the end, in early June, Virtuozzo team did receive a report from the node with the slub_debug enabled. From this report, they understood which object caused memory corruption and also understood the nature of corruption – it was not use-after-free but considerably rarer access-beyond-end-of-object.
This essentially helped in the investigation and a couple of days later Vasily discovered the cause of the issue.