CVEs, Damned CVEs, and Statistics

Joao Correia

September 30, 2024 - Technical Evangelist

Post Views: 17

Is your vulnerability scanner showing zero problems with your Linux systems? If so, it’s probably missing something important. Conversely, if it’s suddenly showing hundreds of vulnerabilities, that’s likely an overstatement too. And if your compliance reports look clean and problem free? Well, you can bet they’re far from accurate.

Vulnerability Scanners and the Data Dilemma

Vulnerability scanners are a critical tool in a cybersecurity team’s arsenal. Their purpose is to provide visibility, offering peace of mind by assessing systems for known security issues. They allow security professionals to track potential threats before they become real problems, ensuring the organization’s systems are secure – or at least that’s the theory.

However, these scanners are far from magical. They don’t make educated guesses or intuitively detect vulnerabilities. Instead, they rely on predefined lists of known issues, the quality and completeness of which significantly impacts the results. And these results can fluctuate dramatically – by orders of magnitude – depending on the data being fed into the scanners.

The Linux Kernel CNA and Its Ripple Effect

A noteworthy development in the cybersecurity space this year has been the establishment of the Linux Kernel CNA (CVE Numbering Authority). We’ve discussed the details in a previous report, but the gist is that every bug in the Linux Kernel now receives a CVE (Common Vulnerabilities and Exposures) identifier. If you follow the Linux Kernel mailing list (good luck keeping up!), you’ll quickly realize just how many bugs are reported daily. This is to be expected given the millions of lines of code in the Kernel.

This change has drastically increased the number of CVEs impacting Linux systems. The sheer volume of Kernel CVEs now being generated means that distributions shipping different kernel versions are affected by far more CVEs than just a few months ago. In fact, the number of CVEs impacting the Linux Kernel has skyrocketed compared to January of this year.

So how does this impact vulnerability scanners? They now have to contend with a massive influx of new CVEs, which presents two problematic outcomes: scanners either flag your systems for hundreds of new CVEs each week, or they ignore the new CVEs entirely.

The Problem of Unscored CVEs

Here’s where things get messy. CVEs need risk scores to be meaningful. But, with the flood of new CVEs, even NVD – the National Vulnerability Database – can’t keep up. Hundreds of CVEs are being publicly disclosed without a risk score, meaning they’re out in the open, available to anyone, but without the necessary context to determine how dangerous they are.

Pause for a moment and let that sink in: hundreds of publicly disclosed vulnerabilities, unscored and unclassified, are floating around. Without a score, it’s incredibly difficult for security teams to prioritize which vulnerabilities to address first.

And, while some Linux distributions have taken it upon themselves to assign scores, this leads to a fragmented landscape where risk ratings can vary wildly – depending on which vendor’s distribution you’re using. The lack of a unified scoring system makes it challenging to determine which threats are real and which are relatively benign.

The Scanner Conundrum: To Ignore or Not to Ignore?

When scanners are fed these unscored CVEs, they have two options: ignore them entirely, or assume the worst and assign them the highest risk until proven otherwise. Neither option is ideal. If ignored, potentially serious vulnerabilities could go unnoticed. If scored as high risk, you might find your systems flagged with an overwhelming number of false positives, sending your security team into a frenzy over nothing.

Thus, the statistics produced by vulnerability scanners – whether they show your system as secure or vulnerable to hundreds of CVEs – are often more of a best guess than a reliable assessment. You might get similar accuracy by flipping a coin.

So, Are Vulnerability Scanners Useless?

Not at all. Vulnerability scanners still provide valuable data, and some of their tests are effective. However, in today’s environment, they shouldn’t be considered the sole source of truth or relied on as definitive evidence of security. The results must be taken with a grain of salt, especially in the context of the current surge of unscored CVEs.

Is There a Solution?

In the short term, no – not until the rate of CVE creation slows down, or a more efficient method for scoring vulnerabilities is developed. A potential improvement lies in the use of OVAL (Open Vulnerability and Assessment Language) definitions. OVAL files help standardize the way systems are checked for vulnerabilities, introducing a level of curation that can improve accuracy.

However, even OVAL files are influenced by the CVE scoring process, meaning they’re still downstream from the same flawed data source. Vulnerability scanners that use OVAL definitions may be more reliable, but they’re not immune to the broader problem of inconsistent or incomplete scoring.

One potential mitigation strategy is to use multiple vulnerability scanners and average out the risk scores. While this approach isn’t perfect, it could help you get closer to an accurate assessment by smoothing out some of the statistical noise. Right now, there’s no definitive solution – just varying degrees of wrong.

Summary