Re: [CVEPRI] CVE accuracy, consistency, stability, and timeliness

[Dave Mann <dmann@bindview.com>]

Well now!

"Steven M. Christey" wrote:
> Pascal Meunier asked:
> Bill Fithen added:

[Dave, fresh from vacation, tips the soap box up on it's side and
steps up...]

I am strongly in favor of MITRE relaxing its analysis with regards
to the formation of candidates.  I also propose vastly streamlining
the the entire set of Content Decisions to a small set (no more than
6) guidelines.  Finally, I propose that when in doubt, CVE err on
the side of greater specificity.  There are several reasons.

I will open with a thoroughly offensive joke. Rant follows.

Seen on a T-shirt with the US Marines' Logo on it: "Kill 'em
all and let God sort 'em out."


1) CVE was founded on the belief that we, as a community, do not
know enough about this space to formalize it to point of agreeing
on a taxonomy or a database.  While I applaud the desire to achieve
consistency with respect to enumeration issues, I think it is
crystal clear that consistency is only achievable if know enough
to formalize things properly.  And if we understood things to
that level, we wouldn't be involved in CVE -- we would be involved
in a joint database effort instead.   The most important things for us
to do from an academic standpoint is to admit the limitations of
our knowledge.

Given how immature our field is, I think it is overreaching to
believe than any decisions we make now will hold up to scrutiny
in the long run.  I reject the assertion that we can achieve greater
consistency by being more careful because I don't believe that
anybody knows enough to decide on consistency in a rational manner.
I think we have only 2 rational choices.  Either we accept that CVE
will contain (possibly annoying) inconsistencies or we give up.


2) Our recent experience with the SANS Priority One Top Ten list
gives us a concrete example of why CVE should put a higher priority
on completeness than on consistency.  The Top Ten list, of which many
of us provided input, was written at such a high level that it was
terribly ambiguous.  For example, when the SANS list identified
cgi sample files, the expected follow-on question on many lips
was certainly, "Which cgi sample files?"  More clarity and meaning
was added to the the SANS list as soon as they incorporated CVE
names.

"Oh. These cgi files."

But all is not perfect.  CVE falls short, literally, with respect
to the SANS list because it does not adequately cover all of the
known issues identified by the SANS list.  Witness the large number
of CAN numbers instead of CVE numbers that are reference to by the
SANS list.  I draw two immediate conclusions from the SANS Priority
One exercise with regards to CVE.
  a) CVE must put a higher priority on timeliness and completeness,
     even at the price of less consistancy.
  b) When in doubt, CVE should strive for greater specificity
     and avoid high level generalization.

3) Speaking as a vendor, CVE has greater value to me the more coverage
it has.  I do not expect one to one mappings to my peers.  CVE is
an enabling technology that makes life easier.  I do not expect, nor do
I need consistancy.  Again, our internal experience with CVE here at
BindView is that the more precision or specificity, the better.


So, I propose that we create a new T-shirt.  The CVE logo with
the following:  "Name 'em all and let the taxonomists sort 'em out."


Dave


--
==============================================================
Dave Mann                ||   e-mail:  dmann@bos.bindview.com
Senior Security Analyst  ||    phone:  508-485-7737   x254
BindView Corporation     ||      fax:  508-485-0737