RE: Counting on CVEs

["Boyle, Stephen V." <sboyle@mitre.org>]

This might sound like splitting hairs, but how “vulnerabilities” are counted may well enter into this. I’m not claiming that CVE is keeping up – both Kent and others have correctly stated reasons and history that apply, and yeah, somebody who’s only looking at somebody’s raw numbers (be they CVE or anything else) is going to ask hard questions, especially when money is hard to come by.

 

Having said that, it bears mentioning that by design, there are always going to be fewer CVEs than there are “vulnerabilities” – it’s kinda one of the key features. J There are also more players in the space than there were a few years ago, each of which has multiple incentives to publish more vulnerabilities than others.

 

Again, I am not saying there’s not a problem – we have to be able to answer honest questions such as the one Kent relayed. But we also have to be mindful of what are real, what counting problems exist in all vulnerability reporting sources, and what that means for CVE.

 

Tx,

Steve

 

From: owner-cve-editorial-board-list@lists.mitre.org [mailto:owner-cve-editorial-board-list@lists.mitre.org] On Behalf Of Kent_Landfield@McAfee.com
Sent: Wednesday, March 07, 2012 2:53 PM
To: cve-editorial-board-list
Subject: Counting on CVEs

 

All,

 

We have problems with CVE reporting that are known issues but which are becoming apparent to many more and could easily undermine the usefulness of CVE identification if left unchecked.

 

We discussed this at the ITSAC conference in the Future of Vulnerability Reporting Workshop and at the follow-on Vulnerability Reporting day at the Software Assurance event a month later. (There were action items out of the latter that I am not aware have been completed…)

 

I have just had a very concerning discussion about the usefulness of CVEs as a means to measure vulnerabilities today and the decay of its value if the trend continues.   The discussion centered around the accuracy of the numbers of CVEs identified compared to those reported in the community as a whole.  If we looked at just the CVE numbers,  it appears that the numbers of vulnerabilities have been dropping since a high in 2008.   This is a rather important error. As we all know, this is not accurate. Vulnerabilities have not been dropping, they are growing, not dropping by 30%.  

 

For the vendor community, these trends have rather concerning potential impacts on us.  First, our vulnerability research databases use CVE as a primary reference.  The CVE Id has been authoritative in the past.  It is used internally as a means to communicate vulnerability record information between fielded products and between research analysis teams.  As the numbers decline, it means we are forced to look elsewhere to provide the identification and communication that CVE provided in the past. More proprietary ids are becoming more the norm.

 

The more serious concern is what it is showing to executives of companies.  

 

"If the vulnerabilities have dropped  30% since 2008, why do I need to be funding the security staff and efforts at the rate I am?  I see that MITRE is reporting an overall drop each year since 2008 but yet you folks keep coming to me saying that the threats are much worse and we may be in the same relative situation we were when malware spiked a couple years past…"

 

For those that actively work in this environment, we know vulnerabilities have not dropped 30% since 2008. Quite the contrary, our internal numbers indicate an increasing trend similar to a 30% rise.  Symantec has also reported a similar trend.

 

One of the major problems is that MITRE funding is not what it should be. On multiple occasions we have been asked to target the classes of products where vulnerabilities are considered the most critical and which sources should be monitored. The view of what to target and monitor gets smaller and smaller as funding is  held level or reduced.

 

At one point the intent of the effort was to cover all published vulnerabilities that could be corroborated in some fashion.  Over the years the reality has set in that this is a very resource intensive operation.  As such the focus of the effort has reduced what is reported on to assure CVEs can be assigned for the types of products  most important to the Editorial Board participants and their sponsor.  This gives an artificial view of the numbers of existing vulnerabilities and that is being recognized outside the vulnerability community.

 

Another problem is the CVE format itself.  There has been an active discussion about the format limitations as well. The CVE format is CVE-YYYY-NNNN. This means that currently we cannot have more than 10,000 CVEs reported in a single year.  At the rates we are seeing internally, we are already there.

 

Then there are the limitations of CVE process in general.  The focus is English only although some non-US vulnerabilities do get assigned from time to time.  CVE does not support the international community and software written for non-English geo-centric areas are not included.  While this has not been a concern for US-only software vendors, there is a great deal of regional software written for major emerging markets.  None of those vulnerabilities are identified by a CVE.  

 

Given these constraints, it seems like it is time to figure out how to address the issues in a more of a creative way.  We know the constraints. Is there something we can do to augment the MITRE staff in certain areas that would help?  I can see the format issue being a rather easy one to attack but it is the coverage issue that is most concerning.  Or we should ignore it and slowly let the value of CVE to the community and vendors decay…

 

Thoughts?

 

Kent Landfield
Director Content Strategy, Architecture and Standards

McAfee | An Intel Company
5000 Headquarters Dr.
Plano, Texas 75024

Direct: +1.972.963.7096 
Mobile: +1.817.637.8026
Web: www.mcafee.com