Re: Counting on CVEs

[Art Manion <amanion@cert.org>]

On 2012-03-08 11:52 , Boyle, Stephen V. wrote:

> This might sound like splitting hairs, but how “vulnerabilities” are
> counted may well enter into this. I’m not claiming that CVE is keeping
> up – both Kent and others have correctly stated reasons and history that
> apply, and yeah, somebody who’s only looking at somebody’s raw numbers
> (be they CVE or anything else) is going to ask hard questions,
> especially when money is hard to come by.
> 
> Having said that, it bears mentioning that by design, there are always
> going to be fewer CVEs than there are “vulnerabilities” – it’s kinda one
> of the key features. JThere are also more players in the space than
> there were a few years ago, each of which has multiple incentives to
> publish more vulnerabilities than others.
> 
> Again, I am not saying there’s not a problem – we have to be able to
> answer honest questions such as the one Kent relayed. But we also have
> to be mindful of what are real, what counting problems exist in all
> vulnerability reporting sources, and what that means for CVE.

The questions remain IMO:

1. What level of abstraction is appropriate for CVE?

2. What level of completeness is appropriate for CVE?

How narrowly do we define "vulnerability," the thing to name/count?

Is there desire/need for an accurate count of vulnerabilities?  OSVDB
either abstracts a little more narrowly than CVE and/or collects more
vulnerabilities, so OSVDB has higher numbers.

If CVE or any other database were to try to name and count all publicly
disclosed vulnerabilities, it would be important to be able to
distinguish between a vulnerability that is one of a dozen XSS bugs in a
PHP web app and a vulnerability that is a straight up stack buffer
overflow in httpd.  Sure, count them all, but be able to say that out of
20K vulnerabilities named this year, 61% were XSS or SQLi in web apps
with low distribution.

I'm guessing at some numbers in the above example, but this is a big
reason IMO that CVE numbers have declined.  Vulnerabilities "worth
tracking with a CVE" have declined, not the total number of
vulnerabilities.  Another way to look at it might be that thee criteria
for "worth tracking with a CVE" has changed.

And we're not even talking about threat or asset values (both of which
have changed over time, and are different depending on your
site/assets), which influence risk.  So a decrease in CVE IDs has little
directly to do with internet risk overall.


 - Art