[CVEPRI] January 18 Editorial Board Teleconference Summary

["Steven M. Christey" <coley@linus.mitre.org>]

CVE Editorial Board Teleconference Summary - January 18, 2001
-------------------------------------------------------------


Participants
------------

Participants in the teleconference were:

Ken Armstrong (EWA-Canada/CanCERT)
Andy Balinsky (Cisco)
Tim Collins (NFR)
Andre Frech (ISS)
Shawn Hernan (CERT/CC)
Scott Lawler (DOD-CERT)
Jim Magdych (NAI/PGP)
David Mann (BindView)
Pascal Meunier (Purdue CERIAS)
Larry Oliver (IBM)
Craig Ozancin (AXENT/Symantec)
Mike Prosser (Symantec)
Kevin Ziese (Cisco)

MITRE participants included Dave Baker, Bob Martin, Margie Zuk, and
Steve Christey.


CVE Content Update
------------------

The Board was updated with the latest statistics for CVE content,
including estimates for the next CVE version.  On January 16, 234
candidates were moved to the Interim Decision phase.  232 of those
candidates have since been added to the new version, 20010122.  There
are now 1309 entries and 815 active candidates, with 185 candidates
needing at least 1 more vote before they can be converted to entries.
The remaining active candidates are affected by unresolved content
decisions, or they have received a REVIEWING, REJECT, or RECAST vote.

Statistics were also provided regarding submissions.  (See
http://cve.mitre.org/board/archives/2000-07/msg00024.html for
background information on the submission process).  There are
currently 10095 submissions, i.e. items from vulnerability databases
submitted by various organizations.  Most of them were obtained in
mid-2000, when 10 organizations provided MITRE with their
vulnerability databases.  Approximately 1400 submissions have been
matched against other submissions, candidates, and entries.  400 of
those submissions are being refined into candidates.

In the past few months, several new CVE content team members have been
added.  Ramsay Key and Jeff Taylor, the newest members, join Barbara
Pease, Jean-Paul Otin, Dave Goldberg, and Dave Baker.


New CVE Content Goals for MITRE
-------------------------------

With the current CVE content team, MITRE projected that all legacy
submissions will be matched by June 15th.  By July 1, submissions for
security issues that were publicized in 1999 will be refined into
candidates.  The remainder of the legacy submissions are expected to
be refined into candidates by the end of 2001.  Another task is to
process the remaining submissions for problems announced in 2000; due
to the nature of the process, some submissions have "slipped" through
the cracks.

By processing all legacy submissions and converting them into
candidates, MITRE would significantly increase the comprehensiveness
of CVE, allowing CVE-compatible products to map 80% or more of their
own vulnerabilities to CVE names.

In general, Board members believed that MITRE's projected deadlines
for legacy submissions were sufficient for their needs.  There was
some discussion regarding how long it takes Board members'
organizations to research and create their own database entry for an
individual vulnerability.  The amount of time varies from minutes to
days (depending on complexity, accuracy of information, whether a
complete exploit has to be constructed, etc.)  However, the Board
agreed that MITRE's estimated review times seemed reasonable.

Since the CVE content team has only solidified in the past few months,
the projections may be inaccurate.  Some members also expressed
concerns that the schedule could slip - or CVE content in general
would suffer - if a critical content team member became sick or left
the project.  While MITRE has fully staffed the CVE content task as
budgeted, it may be insufficient for the amount of work that is
required to process legacy submissions.  Documentation and training is
taking place to minimize the impact of the loss of a content team
member.  Progress will be reviewed at the next face-to-face Editorial
Board meeting in March.

MITRE's main CVE activities were presented to the Board.  In addition
to processing legacy submissions, MITRE is also keeping up-to-date
with new security issues.  Some Board members expressed concerns that
it currently takes too long - sometimes more than a month - to assign
candidates for newly publicized problems.  These delays have occurred
for two main reasons: (a) new security problems have been processed
entirely by Steve Christey, and are thus affected by his other CVE
activities; and (b) there has been a significant increase in the
number of new security issues announced per month.  In 2000, a at
least 1082 new security issues were disclosed, approximately twice as
many as there were in 1998.  The delays will be addressed in two ways.
First, one of the content team members will begin assisting Steve in
creating candidates for new issues.  Also, it is expected that there
will be a continued increase in the number of candidates that are
reserved by organizations for inclusion in the first public
announcement of a new vulnerability.  These two factors should speed
up the process of assigning candidate numbers to new security issues.

In addition to the creation of candidates for new and legacy problems,
MITRE is also increasing the focus on CVE compatibility (by educating
the public, finalizing the requirements, and developing an evaluation
process).  MITRE plans to improve voting support for Editorial Board
members, to make it as easy as possible for them to vote.  MITRE is
also adding software vendor liaisons, who could consult on CVE items
related to their own products.


Editorial Board Voting Status and Issues
----------------------------------------

It was noted that without specific goals for the Editorial Board,
voting on candidates generally is not at the level that is necessary
for handling the workload.  When major goals are set, the Board as a
whole works together to reach those goals, but major goals are only
set about twice a year.

Overall Board participation was reviewed.  Of 38 non-MITRE Board
members, 17 had voted on candidates for new issues that had been
announced in the past year.  12 of those members had been on the Board
for at least 6 months, and 5 of them had voted on less than 50
candidates over that time (which is 10% of the candidates that were
proposed).  While some Board members play other roles in the Board and
are not expected to vote, the percentage of voting members - and the
average level of activity - is lower than preferred, as the Editorial
Board has been regarded as an organization of highly technical
members.

In addition, there has been an overall decrease in the number of votes
per candidate, and there have been more NOOPs ("No Opinion" votes).
This has resulted in increased noise in the CVE entries that are
produced; for example, a duplicate CVE entry was inadvertently added
to CVE, partially because it only had the minimum number of reviewers.
In some cases, some candidates with vendor acknowledgement got a vote
from a MITRE Board member (Dave Baker) and only one non-MITRE person.
There has also been a lack of consistency of voting for organizations
who have 2 members on the Editorial Board.  Use of the voting web site
has also declined since it was first made available to Board members
in Fall 2000.

Overall, it is estimated that at least 5 different voters are needed
per candidate, to ensure that it is properly reviewed.  While
statistics are not available at this time, it is believed that the
Board is not reaching this level of participation in voting.

Some of the newest Board members said that they had not been provided
with the supporting documentation for voting, such as information on
the voting web site.  Those members will be provided with the voting
documentation, and it will be included in the support package for any
new members that join.

Some of the decline in voting is also due to the more stringent voting
guidance that was created in the fall, as documented in the VOTE
content decision (see
http://cve.mitre.org/board/archives/2000-10/msg00000.html).  The
guidance suggests that a Board member should have good confidence that
a reported vulnerability is real.  Some Board members said that they
were not voting as often because of the more stringent voting
guidelines; for example, some Board members were not comfortable in
voting on candidates that they did not verify personally.  As
currently written, the guidance does not include specific criteria for
determining when a candidate is real, so different Board members apply
those criteria in different ways.

Some criteria for establishing the veracity of a candidate were
suggested.  A Board member could be reasonably confident that a
candidate is a legitimate security problem if: (a) they verify the
problem themselves; (b) the affected vendor has acknowledged that the
problem exists; (c) someone they trust has verified the problem; or
(d) there has been confirmation by independent sources.

The Board decided to have a more detailed discussion of voting
guidelines and practices at the face-to-face meeting in March.
Voting-related topics during that meeting will include: (a)
formalizing voting requirements (e.g. what percentage of candidates a
Board member should be "expected" to review); (b) making Board
members' voting records more easily accessible to the public; (c)
establishing more consistent guidelines for all Board members to
follow when voting; and (d) how confident the Board should be in the
veracity of candidates, before they are converted into entries and
added them to the official CVE list (this in turn affects how
stringent the voting guidelines should be.)

Board members also discussed the possibility of providing custom
voting capabilities to individual members, outside of the normal
process of using the standard e-mail ballots or the voting web site.
MITRE will work with interested individual members to develop ad hoc
voting processes that make it as easy as possible for them to vote.


Content Goals for the Editorial Board
-------------------------------------

Participants discussed the next set of CVE content goals for the
Editorial Board.  Members agreed to expand CVE to 1600 entries by
June.  It is also hoped that Board members will vote more actively and
increase the average number of voters per candidate.

Given the changes in voting practices that may take place after the
face-to-face meeting in March, it was agreed that setting more
long-term goals would not be practical.


Confidence Levels
-----------------

In November 2000, MITRE proposed the idea of attaching "confidence
levels" to CVE candidates and entries.  (See
http://cve.mitre.org/board/archives/2000-11/msg00006.html and
http://cve.mitre.org/board/archives/2000-11/msg00007.html).  However,
in the teleconference, MITRE stated that they would not extend CVE to
include such information.  If they had been adopted, confidence levels
could have simplified Editorial Board voting practices while making
the "quality" of CVE information more explicit to users of CVE.
Several Board members were supportive of this approach.  However,
after offline discussion with some organizations, it was decided that
it was not appropriate to extend CVE to include confidence levels.
The primary motivator was that it potentially competed with existing
or future vulnerability databases.  Since CVE must minimize
competition (real or perceived) with other efforts as much as
possible, MITRE decided not to adopt confidence levels.  However, the
concept may be proposed to the public in another context, outside of
CVE.  In addition, to address potential public misconceptions of the
process by which CVE entries are approved, the voting rules and
guidance could be made more publicly accessible.

Confidence levels could be useful to Board members who vote on
candidates.  For example, one member may have been able to replicate a
reported vulnerability; if other members know this, then they might be
more comfortable voting to accept a candidate, even if they haven't
replicated it themselves.  MITRE will investigate ways of making this
information more easily available to Board members, though the
information will remain private.


CVE Entry Deprecation Process
-----------------------------

While the CVE candidate review process is designed to minimize the
number of entries that are erroneously created, there is a risk that
an entry is improperly assigned.  For example, a duplicate entry may
be added, or an existing entry may need to be split into different
entries, or merged with other entries.  When such a change is needed,
a CVE entry may need to be deprecated.  In the face-to-face meeting in
August 2000, the Editorial Board had some initial discussions on the
deprecation process.

Deprecation was not necessary until recently, when a duplicate CVE
entry was inadvertently added to the CVE list.  The following process
for handling deprecation was presented to the Board; it will be
followed when an entry must be deprecated.

1) Initially, entries that have been marked for deprecation enter the
   "Reassessment" phase.

2) The CVE Editor sends an email to the Editorial Board to identify
   those entries that have been targeted for deprecation.  For each
   entry, the reasons for deprecation will be noted.

3) The Board is provided given a short period of time to review the
   reasons for deprecation, at least 4 business days.

4) The CVE Editor makes a Final Decision to deprecate the entry
   (barring any feedback by Board members).

5) The description of the entry is modified to say that it has been
   deprecated, and it also provides the reason for deprecation.  If
   the entry is a duplicate, then the correct CVE entry or candidate
   is also identified.  The description will contain keywords that
   allow users to quickly find all deprecated entries.

6) The deprecated entry is noted in the CVE difference report, which
   compares one CVE version with the previous version.

7) The entry remains in the CVE list, in case CVE-compatible tools
   still use the deprecated entry.

This process will be followed starting with the next version of CVE.
A different way of identifying deprecation status would be to add
another field to CVE.  That approach will be considered if the current
approach is not sufficient.


CVE Entry Modification Process
------------------------------

Sometimes a CVE entry needs to be modified.  For example, the
description may contain some minor inaccuracies, or additional
references may need to be added.  In the past, such modifications were
made by the CVE Editor without approval by Board members.  However, at
the August 2000 meeting, some Board members suggested that a short
review period could reduce the possibility that inappropriate
modifications would be made to CVE entries.

The following process for entry modification was presented to the
Board.

1) Initially, entries that have been marked for modification enter the
   "Reassessment" phase.

2) The CVE Editor sends an email to the Editorial Board to identify
   the entries that are being modified, providing details of the
   planned modifications.

3) The Board is given a short period of time to review the reasons for
   modification, at least 4 business days.

4) The CVE Editor makes a Final Decision to modify the entries
   (barring any feedback by Board members).

5) The modifications are made in the next CVE version.

6) The modifications are noted in the CVE difference report, which
   compares one CVE version with the previous version.


Candidate Rejection Process
---------------------------

In some cases, a candidate may need to be rejected.  For example, if a
problem is reported, it may be discovered that the reported problem is
not real; or, the candidate may be a duplicate of another candidate or
entry.

The following process for candidate rejection was presented to the
Editorial Board.

1) The CVE Editor identifies the candidates to be rejected, and moves
   them to the Interim Decision phase.

2) The Editor sends an email to the Editorial Board to notify them
   about the candidates that are slated for rejection.  The Editor
   annotates the voting record to specify the reason for rejection.

3) Board members the normal review period to review the candidates,
   typically 4 business days.

4) The CVE Editor makes a Final Decision to reject the candidates.

5) If a candidate is rejected, then its description is changed to
   state that it has been rejected.  The description will contain
   keywords that allow users to quickly find all rejected candidates.
   If the candidate is a duplicate, then the correct entry or
   candidate is also specified.

6) The rejected candidates are identified in a separate report,
   possibly as part of a CVE version difference report.

7) The candidate will continue to be included in the candidates list,
   as downloaded from the CVE web site.  This will allow users to
   understand the status change.

It was suggested that it might be important to keep the description
for some candidates intact.  For example, suppose a security issue is
reported and assigned a candidate number, but it is proven to be
incorrect later on.  If users search the candidates list for the
issue, then they should be able to obtain the candidate name and see
that the candidate has been rejected.  This issue will be revisited at
a later time; currently, the rejection process will only be applied to
duplicate candidates.


Candidate Reservation
---------------------

Candidate reservation is the process by which individuals or
organizations obtain candidate numbers so that they can include them
in the initial public disclosure of vulnerabilities.  The number of
reserved candidates has increased steadily.  Approximately 65
candidate numbers have been reserved, most of them by leading security
organizations.  (See
http://cve.mitre.org/board/archives/2000-05/msg00178.html for some
early rationales for this approach).  As the use of candidate
reservation grows, it will make candidate numbers available in a more
timely fashion.  As more and more organizations adopt it, this may
encourage other organizations to use candidates in their advisories.

While candidate reservation is available to anyone who requests it,
MITRE is not actively publicizing this capability yet.  To provide
rapid response on a larger volume of candidate number requests,
members of the CVE content team need to gain additional experience so
that they can apply CVE content decisions to determine how many
candidate numbers must be reserved.  Also, the organizations who
currently reserve candidates work closely with the affected vendor
before release, and they are able to clearly describe those
vulnerabilities.  This will change as more people request candidates
without providing sufficient information or consulting the vendor.
This in turn would increase the amount of noise in reserved
candidates, as well as the amount of non-public vulnerability
information that is made available to MITRE.  It could interfere with
the current vulnerability disclosure process and/or force MITRE into a
CERT-like role of an intermediary between the discloser, the vendor,
and the public.

One way of minimizing the amount of information that is provided to
MITRE, while increasing responsiveness and decreasing the impact on
the current disclosure process, is to link candidate reservation more
closely to the organizations that are currently involved in the
disclosure process.  This could include major software vendors and
neutral third parties who act as intermediaries between disclosers and
vendors, e.g. CER/CC or SecurityFocus' VulnHelp service.  Some major
operating system vendors are beginning to include candidate numbers in
advisories.  MITRE has begun consulting with some OS vendors to
determine the best way to make candidates available in their security
advisories, including initial public announcements.  If vendors or
neutral third parties are given a pool of candidate numbers to assign
to reported problems, then candidates could be announced without
MITRE's direct involvement.  However, there is an increased risk that
multiple candidate numbers could be assigned to the same issue, or a
candidate might be assigned at the wrong level of abstraction.  Other
complexities exist, such as ensuring that the discloser and all
affected vendors use the same candidate number, and reducing the
amount of information available to MITRE while reliably producing
candidates.

Candidate reservation issues will be discussed in more detail at the
March meeting.


Other Issues
------------

Following are some other issues that were discussed or presented.

1) MITRE will be focusing more on CVE compatibility this year,
   e.g. expanding the requirements document for new types of products,
   finalizing the evaluation process, and working more closely with
   vendors.  Bob Martin of MITRE will be leading this task.

2) MITRE is continuing to form the CVE Advisory Council, a group of
   CIO-level managers who provide funding and high-level guidance for
   the CVE Initiative.  The next Council meeting is on January 25th.

3) MITRE is working to establish liaisons with software vendors.
   These liaisons will not be a formal part of the Editorial Board;
   however, they will be available to consult with the Board on
   technical security issues related to their own products.

4) MITRE is re-evaluating the requirements for Editorial Board
   membership.  In the past, the Board has been an informal
   organization.  As it grows, however, it becomes more important to
   be precise about the roles and responsibilities, and the expected
   tasks, for each member.  Board membership requirements will be
   reviewed at the March meeting.

5) Board members discussed possible dates for the next face-to-face
   meeting.  The meeting will be hosted by Cisco in mid-March.  In
   addition to the topics mentioned in this summary, other points of
   discussion will include CVE compatibility, a progress report on the
   IDS-specific Common Intrusion Event List (CIEL), and new approaches
   to critical CVE content decisions.