Tech Support
Documentation
Login
Contact Us
TuxCare BlogThe Great Kernel CVE Flood of 2024
by Joao Correia
December 12, 2024 - Technical Evangelist
“We’re just doing what cve.org wants us to do” was repeated multiple times by Greg K-H in a recent presentation. Earlier this year, the Kernel team’s new CNA (CVE Numbering Authority) status surprised many in the security space. Not only was the previous stance completely opposed to the CVE system – “Burn them down!” was a proposed solution – but they were using a different approach to track vulnerabilities (commit ids) and not making them public as security flaws in the first place.
The CNA change has had a ripple effect across the industry, for reasons that are becoming more and more obvious: not only are there more CVEs, but the massive numbers make tracking, analyzing, and identifying the important ones much harder.
But this has all been discussed previously by us (and by many others in the industry). There’s even a report we released a few months back, if you’re so inclined.
What’s interesting is the viewpoint from the Kernel team itself about this issue, and, in this article, we’re going to dissect the latest presentation and identify the problem that was sidestepped during that presentation.
All the quoted text below is taken from the presentation “CVEs are alive, but do not panic!” by Greg K-H, maintaining as much of the context as possible.
The Elephant in the Room
Let’s start with some numbers. These are provided explicitly in the presentation. The Kernel CNA is currently (as of October 2024) issuing an average of 55 new CVEs per week. This is slightly less than the 60 per week that happened during the first few months of its CNA activity. However, in previous years, there were, according to the presentation, about 20 CVEs per quarter.
The initial policy put forward by the Kernel CNA was interpreted as an every-bug-gets-a-cve approach. This has now been further expanded in that only about 20% of bugs actually get one – and that gives us 55 per week. “That’s a lot.” Tracking the Linux Kernel mailing list, that feels about right.
The process goes like this: for every stable commit, there are (at least) three persons looking at it and deciding if it merits a CVE or not. This gets publicly discussed when there is no consensus. If such consensus is reached, a CVE is assigned.
When a CVE is assigned for a particular bug, it can still be revoked after the fact if convincing arguments are presented. This accounts for 1% to 2% of assigned CVEs.
The Kernel CNA Is “Not #1”
…except that it is. Because of the (very large) numbers involved, and obviously to address criticism, the presentation includes a list of other projects and the number of weekly CVEs they generate. WordPress, with 112 per week, is at the top – with the caveat that it’s not really wordpress, as it includes all the plugins and themes, so it’s really not 112 per week. MITRE is shown with 95 per week, but that is not a single project either. Windows (11-12), Chrome (6), macOS (6), iOS (2-3) are also on the list. Open source vs closed source means these figures can’t be independently verified for accuracy, but having the kernel accrue more CVEs than any other individual project out there, and more than all competing OSes put together, might tell more about possible problems with the process than with the kernel itself.
So Why Did this Happen?
The stated reason for starting to issue CVEs was twofold: first, cve.org was now permitting open-source projects to have their own CNA, and, secondly, there are laws and regulations in place that require disclosure of vulnerabilities across different jurisdictions (with more to follow).
“We are required to do this” [referring to issuing CVEs], so it’s not like the proverbial genie can be put back in the bottle.
What Constitutes a Vulnerability?
One of the main differences between the Kernel CNA and other projects’ CNAs is, as mentioned, the large number of issued CVEs. This boils down to the interpretation of the definition of CVE by cve.org. Per the Kernel team’s reading, the following bugs mandate the creation of a new CVE (quoted from the presentation):
- ”Any user-triggerable crash or reboot
- Memory use-after-free / leak / overflow
- Incorrect boundary checks
- Denial of service
- Logic errors
- Lots of other things”
The following are specifically NOT considered vulnerabilities and won’t get CVEs:
- ”Data corruption / loss
- Performance issues
- Bugfixes that are not externally triggered
- No hardware CVEs”
Specifically the first point in the second list, data corruption, merits a longer explanation during the presentation, and, quite frankly, not considering potential data loss bugs as security problems ignores the potential associated denial-of-service attacks associated with this (as was correctly pointed out by a member of the audience during Q&A). The reason? “We’re following cve.org guidelines. It’s their problem, not ours.” Not a very helpful position.
“Good luck figuring this out”.
The Big Problem with Enterprise Distributions
Representatives of different distribution vendors attended the presentation, and during Q&A they made their view on the matter very clear – they were not happy about it, to put it mildly.
The reasoning here is that, if you’re consuming the kernel and integrating it into your environment, then you (should) have a good understanding of what you’re using, and so it would be less excruciating to identify the CVEs that are actually meaningful to your particular environment – you don’t need to worry about code that is never used or loaded – so a considerable portion of the 55 CVEs per week doesn’t really matter in each particular instance.
However, distribution vendors are in a position closer to the kernel team than to end users. A distribution can be used in an infinite amount of different ways and situations, and, just like the Kernel team, the vendor can’t pretend to know which parts of the kernel will be relevant or not. As a result, they have to address all the vulnerabilities that come their way. This trickles downstream to companies who work on top of the distribution vendor’s releases.
So each of those 55 CVEs per week has to be analyzed, scored, fixed, tested, and released to enterprise customers. If you were looking for a reason why vendor security bulletins now include hundreds of CVEs, it’s because they get lumped together every few weeks.
Still, it would simply be a matter of transposing the fixes from upstream to downstream and calling it a day, right? Well, there’s more to it than that.
CVEs assigned by the Kernel CNA only reference the actual fix commit. If there is a whole chain of other patches that are necessary for the fix to apply, or work correctly, that’s left as an exercise for the reader.
They are also “not tested independently, only as a block.” This becomes useless as soon as a distribution applies any customization, as all the tests then need to be repeated.
“The Enterprise Distros do have a problem, that’s self inflicted, and I have no sympathy” [talking about backported fixes to older kernel versions that are shipped and are supported by distribution vendors]. Indeed.
“It’s not my problem.”
The End User/Organization
“[You need to update to the very latest kernel] and reboot, or triage all the fixes, apply the relevant ones and reboot. You can use live patching.” In the same breath, he acknowledges that many can’t, as their infrastructure can’t handle the number of reboots. So triage 55 CVEs per week, every week, or take the latest version. This is a real-world pyrrhic choice that becomes extremely costly in an enterprise environment.
The third and, all things considered, only actionable alternative is to live patch your systems.
The State of Things
“We are doing exactly what cve.org asked us to do” seems to be the default stance in defending the current process. Accepting that at face value, and that “we are not causing a denial of service” [on the CVE process] is also not the intention. But the actual real-world situation faced at many companies, distribution vendors, and cybersecurity companies, and the compliance and security problems that come with having systems flagged for 55 new vulnerabilities every week, doesn’t really aid in accepting the current state of affairs as anything other than (intended or accidental) malicious compliance.
The tremendous work done by the Kernel team, and particularly the small Kernel CNA team, deserved better, but is unfortunately causing too much grief and suffering on thousands of security professionals whose entire ecosystem is now flooded with new, often unscored, issues that it just added more of a burden to an already stressful environment.
“It’s causing some churn and some bumps in the ecosystem. I know I’m going to be quoted on this.” No kidding.
Note: Thank you to Greg K-H for the transparency into the process and for the look behind the curtain.
