Re: procedure for penalizing or revoking CNA status?

["Steven M. Christey" <coley@mitre.org>]

On Thu, 25 Sep 2014, jericho wrote:

> For the rest of the board, there has been an increasing reason to better 
> monitor and restict CVE assignment. Both from researchers requesting them, 
> and from CNAs who don't understand CVE or the abstraction process.

The former issue - about researchers requesting CVEs - is separable from 
how CNAs manage CVEs.  I will address the latter here.  (For the former, 
the increasing variety of researcher skill levels, combined with our 
desire to minimize access to 0-day information, currently allows certain 
issues to go undetected during reservation.  We have been aware of this 
and related problems, and our CNA team will be developing various 
strategies to minimize such errors.)

Some context for CNA-related errors: traditionally, we've had 
approximately a 0.5% REJECT rate for CVEs overall, but that percentage has 
gone up in recent years, although I don't track these stats regularly or 
precisely (yet).  While I personally dislike REJECTs, the 0.5% rate 
doesn't indicate a systemic problem.  But since the raw number of CVE 
assignments has also risen along with the rate, the raw number of REJECTs 
has increased noticeably.  REJECTs, for us and I believe for many CVE 
consumers, can cause confusion and be time-consuming to resolve.

We do not have any formal procedures for warning, penalizing, and/or 
revoking CNA status, but we agree that we should develop some.  One issue 
is that things have gotten much more complex, and what might appear to be 
a CNA error could in fact be due to limitations of the CNA process, many 
of which were discussed in the early days of CVE, if I recall correctly. 
When developing procedures, we also need to ensure that any disciplinary 
measures - when necessary - are not out of balance with the offense.

However, we also need to be clear on what is causing the errors.  The 
errors that occur are rarely due to carelesness.  For example, we've 
learned that over time, people's jobs (naturally) shift; and the original 
technical lead for a CNA might move to a different role, and the 
replacement is not as well-trained.  As another example, there are 
researchers who contact multiple CNAs for CVEs and effectively introduce 
duplicates that way (not maliciously, as far as I can tell); many 
researchers, especially those new to the industry, don't really understand 
how CVE works, and are not necessarily diligent in reading our fairly 
extensive documentation.  As a third example, the significant media 
attention and urgency given to some issues, along with non-coordinated 
disclosure, introduces room for error.  Incomplete *disclosure* 
coordination happened with both Heartbleed and Shellshock, and was a 
factor in the confusion - for which CVE was a symptom and not a cause 
(and, partially, a cure because we did REJECTs pretty quickly.)  There are 
many more situations besides these.

These types of errors become more pronounced when dealing with higher 
volumes of CVEs, with more and more people communicating using CVE, since 
the "network" of CVE-speaking parties effectively becomes more complex.

In the coming months, we will improve our tracking for REJECTs and why 
they happen; consult more closely with CNAs; and consult with the Board on 
ways forward.

- Steve