[CVEPRI] CVE Compatibility Requirements - for Board meeting

["Steven M. Christey" <coley@linus.mitre.org>]

All:

Below is a draft set of requirements and recommendations for CVE
Compatibility.  We will be reviewing these requirements during the
Board meeting, on Thursday.  If you get a chance, please familiarize
yourself with them ahead of time.

There are several considerations that must be made while reviewing
these requirements.  First and foremost, they should serve as explicit
guidance to tool and database developers ("repository owners") who are
seeking CVE compatibility.  Second, with CVE and compatibility still
new, we must be able to be flexible with respect to what it means to
be compatible - and how it is to be measured - at least in this early
stage.  Third, making requirements too stringent at this stage could
prevent the adoption of CVE in tools or databases by making it too
complicated, expensive, or administratively difficult for someone to
achieve compatibility.

Note that we at MITRE believe that there is room for a much stronger
notion of "CVE Certification" or "CVE Compliance" which may have more
stringent requirements and be more difficult to achieve.  Some Board
members have effectively requested this already.  It is unclear as to
whether MITRE is the appropriate organization to provide such an
"Underwriters Laboratories" capability at this time.  Therefore we
should focus on keeping CVE Compatibility as a more accessible
concept, then at a later time move forward to more restrictive
requirements if necessary.

Finally, the ever-changing nature of CVE poses complications for a
product to "stay" compatible, and for the review authority (currently
MITRE) to continually "re-evaluate" the accuracy of mappings each time
a new CVE version comes out.  The fundamental issue is whether CVE
Compatibility should be directly associated with a single CVE version,
or if it should be regarded more as a continual process.  There are
pros and cons with both approaches.

- Steve




Requirements and Recommendations for CVE Compatibility
------------------------------------------------------
Steve Christey
The MITRE Corporation
coley@mitre.org
CVE web site: http://cve.mitre.org

Document version: 0.3
Date: March 7, 2000
Expires: June 4, 2000


*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***
*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***
*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***

This is a draft report and does not represent an official position of
the MITRE Corporation.  (c) 2000, The MITRE Corporation.  All rights
reserved.  Permission is granted to redistribute this document if this
paragraph is not removed.  This draft is subject to change without
notice.

*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***
*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***
*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***


*****************************
Table of Contents
*****************************
1. Process for Determining CVE Compatibility
2. Definitions
3. Prerequisites
4. High-Level Requirements
5. Requirements with respect to CVE Versions
6. Requirements for Repositories that Use Candidate Numbers
7. Additional Requirements for Web Sites
8. Additional Requirements for Security Tools
9. Additional Requirements for Other Repositories
10. Accuracy Requirements
11. Good Faith Requirements
12. Revocation of CVE Compatibility


***********************************************************
1. Process for Determining CVE Compatibility
***********************************************************

For a security product such as a tool or database to be CVE
compatible, it MUST satisfy the requirements that are outlined in this
document.

If you want to claim that your product is CVE Compatible, you must do
the following.

1) Review this document on requirements and recommendations for CVE
compatibility, and ensure that your product (or a future version) can
satisfy them.  Have your product development team review the document
"Technical Details for CVE Design and Maintenance," which includes
information on CVE mappings and compatibility.

2) Make a public declaration of your intention to make your product
CVE compatible.  Declarations can be added to the CVE web site.  See
the "CVE Compatible Products" page at http://cve.mitre.org for
details, and send your declaration to cve@mitre.org.

3) A CVE team member will contact you to obtain further details.

4) As your product makes concrete steps toward achieving
compatibility, e.g. by adding new functionality, you may send updates
to the CVE web site by emailing cve@mitre.org.

5) MITRE (or other Review Authorities) will conduct an informal review
of your product to *review* if it is CVE-compatible.  If the product
satisfies the requirements described in this document, MITRE will
*approve* the usage of the CVE compatibility term, and grant limited
permission for you to use the trademarked CVE logo and
CVE-Compatibility phrase.


*****************************
2. Definitions
*****************************

Security Element - a database record, email message, security
advisory, assessment probe, signature, etc., which is related to
information system security information.

Repository - a collection of security elements, e.g. a database,
security tool, or web site

Owner - the owner or maintainer of the repository

Map/Mapping - the specification of relationships between security
elements in a repository and the CVE entries or candidates that are
related to those elements.

Review - the process of determining whether a repository is CVE
compatible.

Review Version - the version of CVE that is being used for determining
compatibility of a repository.

Review Authority - an entity that performs a Review.  MITRE is the
only Review Authority at this time.  Currently, there is no process
for assigning other Review Authorities.


**********************************************************
3. Prerequisites
**********************************************************

1) The repository owner MUST be a valid legal entity, e.g. a
corporation or a specific individual.

2) The Owner MUST provide MITRE with free access to the repository.
This allows MITRE to (a) review the mapping for accuracy, and (b)
identify repository elements that must be added to CVE.

3) The repository MUST provide additional value or information beyond
that which is provided in CVE itself (i.e. name, description,
references, and associated candidate data).


**********************************************************
4. High-Level Requirements
**********************************************************

These are the high-level requirements for all repositories.  They are
described in detail in later sections.

1) The repository MUST allow users to locate security elements using
CVE names ("CVE-Searchable").

2) The repository MUST be able to include CVE names when it presents
security elements to the user ("CVE-Reportable" or "CVE-Output").

3) The repository MUST properly identify the CVE Review Version that
the Review Authority has used for determining compatibility.

4) The repository MUST satisfy the Accuracy requirements, as specified
below.

5) The repository MUST satisfy the Good Faith requirements, as
specified below.

6) The repository MUST include a prominent reference to the CVE web
site (http://cve.mitre.org)


The repository is NOT REQUIRED to do any of the following:
  - use the same descriptions or references as CVE
  - use CVE candidate numbers
  - include every CVE entry in its repository


*********************************************************************
5. Requirements with respect to CVE Versions
*********************************************************************

This set of requirements poses some challenges.  On one hand, it is
important that end users know how "up-to-date" a repository is with
respect to its mappings, and the CVE version can play a role in this.
However, it is not yet defined how Review Versions and Reference
Versions should be assigned.  As CVE versions change on a regular
basis, it must be considered whether CVE-Compatibility should be
re-evaluated for each version, or if it should be regarded more as a
"process" during which a full-fledged review is only performed
periodically.

1) The repository MUST be reviewed for CVE compatibility with respect
to a specific CVE version, i.e. the Review Version.

2) The repository MUST clearly identify which Review Version was used
to determine compatibility.

3) The repository SHOULD be compatible with a CVE version that is an
official CVE "Reference Version" as announced by MITRE.  On a periodic
basis (approximately once per quarter), MITRE and/or the CVE Editorial
Board will announce that a specific CVE version is to be used as a
Reference Version.

4) If the Owner believes that the repository would satisfy
compatibility requirements for CVE versions that are more recent than
the Review Version, then the Owner MAY report that it is "up-to-date"
with respect to the more recent version.  The Owner MUST NOT claim
that it is compatible with the more recent version.

5) If the repository uses candidates, then it SHOULD indicate the
release date associated with the candidate information.


*********************************************************************
6. Requirements for Repositories that Use Candidate Numbers
*********************************************************************

A repository MAY include CVE candidate numbers.  It is NOT required to
do so.  CVE Compatibility is only established with respect to the
official CVE.  Candidates are not expected to become a part of the
requirements for CVE compatibility.  In cases where repositories use
candidtaes, however, certain recommendations must be made.

If the repository uses candidate numbers, then:

1) The repository MUST clearly indicate to the user that the candidate
number is not an accepted CVE entry.  The use of the CAN- prefix
itself is sufficient to indicate candidate status.  However, the
repository SHOULD include additional supporting text that describes
how candidates are different than official entries.  The repository
SHOULD include the text "(under review)" after the candidate number
when it is used in natural language text descriptions.

2) If a candidate has become an official CVE entry within the Review
Version, the repository owner SHOULD replace that candidate number
with the associated CVE number/numbers for all affected security
elements.  In other words, the owner must keep up-to-date with respect
to candidates that become official entries.

3) If a user performs a search using YYYY-NNNN, the repository SHOULD
return the corresponding CAN-YYYY-NNNN or CVE-YYYY-NNNN.

4) If the repository contains the official entry CVE-YYYY-NNNN, but
the user searches using CAN-YYYY-NNNN, then the repository SHOULD
return CVE-YYYY-NNNN, and vice versa.


**********************************************************
7. Additional Requirements for Web Sites
**********************************************************

1) The web site MUST allow a user to type in a CVE name and retrieve
the related security elements from the repository.  Alternately, the
site MAY satisfy this requirement by providing a web page which links
CVE numbers to the related elements.

2) A web page that provides details for an individual security element
MUST identify the CVE entry/entries that map to that element.  If web
pages are not provided for individual elements, then the CVE name MUST
be associated with the security element in some other fashion.

3) A URL "template" SHOULD be provided which allows a computer program
to easily construct a link that accesses the search capability as
outlined in (1).  This will allow CVE-compatible web sites to easily
link to each other, and reduce the effort of link maintenance.
Examples:

  http://www.example.com/cve/my-db-cve-name.cgi?name=CVE-YYYY-NNNN
  http://www.example.com/cve/CVE-YYYY-NNNN.html

If the URL template is a CGI program, the program MUST accept the
"GET" method.

4) The web site SHOULD use approved "generic" entries in the same way
that normal CVE names are.

5) The web site MAY link to specific CVE entries on the official CVE
web site.  The URL template is:

   http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-YYYY-NNNN



**********************************************************
8. Additional Requirements for Security Tools
**********************************************************

A security tool is a software application or device that examines a
host or network and produces information that is related to
vulnerabilities or exposures, e.g. an assessment tool, intrusion
detection system, etc.  A "task" is a security tool's probe, check,
signature, etc. which performs some action that produces security
information.

1) The tool MUST allow the user to specify a set of tasks by inputting
a file of CVE names.

2) For any report which lists individual vulnerabilities or exposures,
the tool MUST allow the user to ensure that CVE names may be included
in that report.

3) The interface of the tool SHOULD allow the user to browse and
select a set of tasks using individual CVE names.

4) If the tool does not have a task that is associated with a CVE name
as specified by the user in (1) or (3) above, then the tool SHOULD
notify the user that it cannot perform the associated task.

5) The tool SHOULD allow the user to list all tasks in the tool that
are related to CVE entries, without selecting or deselecting them.

6) If the tool provides a repository which is directly accessible to
the user, then that repository MUST satisfy the requirements related
to repositories.

7) If the tool generates web pages, then those web pages MUST satisfy
the requirements for "Web Sites" as described above.


**********************************************************
9. Additional Requirements for Other Repositories
**********************************************************

Requirement (1) is an attempt to characterize what "CVE-Searchable"
and "CVE-Output" might mean for other repositories besides web sites
and tools.  Requirement (2) is less well-defined because it is unclear
as to what CVE-Compatibility may mean in less "traditional"
repositories.

1) If a GUI is provided with the repository, then:
   - A search capability MUST be provided that allows a user to use a
     CVE name and retrieve the related security elements from the
     repository.
   - A window/dialog that provides details for an individual security
     element MUST allow the user to identify the CVE entry/entries
     that map to that element.
   - A user SHOULD NOT be allowed to change the map between a security
     element and the related CVE entry/entries.

2) If a GUI is NOT provided with the repository, then the repository
SHOULD be structured so that:
     - A query or other programmed search mechanism can locate
       related security elements when provided with a CVE name.
     - The CVE name(s) for a specific security element can be accessed
       from each element using a query or other programmed search
       mechanism.
     - If a repository provides a mapping to the end user, then it
       is regarded as satisfying these two requirements.


**********************************************************
10. Accuracy Requirements
**********************************************************

A Review Authority is responsible for reviewing the accuracy of a
repository.  CVE Compatibility only facilitates data sharing when the
mappings are accurate.  It is an open question as to how to define
"accurate" and how to verify accuracy without establishing a
full-fledged "Underwriters Laboratories" organization that performs
certification, which could be resource-intensive.


Review Authority Requirements
-----------------------------

1) If the Review Authority will only evaluate a portion of the
repository for accuracy, then:

  - it MUST define a Sample Size, which is the percentage of security
    elements and/or the number of elements to be examined by the
    Review Authority.

  - it MUST publicize the sampling method.  The resulting sample of
    the repository is referred to as the Review Sample.  The Review
    Authority MAY use non-random sampling techniques.

  - it MUST define a Minimum Accuracy Percentage, which is the
    percentage of the Review Sample for which the map to CVE is
    accurate.  The Minimum Accuracy Percentage SHOULD be 80% or
    higher.

  - it MUST define a Minimum Map Percentage, which is the percentage
    of the repository that has been mapped to CVE.  The Minimum Map
    Percentage SHOULD be 80% or higher.

3) The Review Authority SHOULD use the same sampling method for any
repository that is compatible with the Review Version.


Repository Requirements
-----------------------

1) The repository owner MUST provide the Review Authority with access
to the repository for review.

2) The Mapping Completeness of the repository is measured by
determining the percentage of security elements in the repository that
are mapped to CVE identifiers, i.e.:

   - the CVE name/names of related entry/entries
   - a GENERIC specifier which is approved for use for that Review
     Version, except GENERIC-UNMAPPED.
   - a blank field which is equivalent to an approved GENERIC
     specifier

The Mapping Completeness MUST meet or exceed the Minimum Map
Percentage.

The Review Authority MAY choose to determine the completeness of a
portion of the repository.  The Review Authority MAY use the Review
Sample to determine completeness.

3) The Accuracy of the Review Sample is measured by determining the
percentage of security elements in the sample that accurately
reference the correct CVE and GENERIC identifiers.  The Accuracy must
meet or exceed the Minimum Accuracy Percentage.


**********************************************************
11. Good Faith Requirements
**********************************************************

Good Faith is essential, especially in the early stages when CVE
Compatibility is still being refined as a concept.  However, these
requirements do not provide a means of "enforcing" Good Faith, because
there is no action that an owner MUST do.

1) The repository owner SHOULD provide a technical POC who is
qualified to answer questions related to the accuracy, maintenance, or
construction of the mapping.  The POC SHOULD be made available to the
Review Authority and to the repository owner's users.

2) The repository owner SHOULD correct errors in mappings in a timely
fashion, i.e. within one month or the repository's average "update
cycle" in the past year, whichever is greater.  (Example: a tool that
provides updates every 3 months would be expected to correct a mapping
error within 3 months of the notification of the error).

3) The repository owner SHOULD prepare and sign a statement that, to
the best of their knowledge, the owner has not provided any false
information in the mapping.


Security tools also have the following requirement.

4) The tool vendor SHOULD sign a statement which warrants that, for
each task that is associated with a CVE entry, all of the following
are true:
   - the rate of false positives is less than 100% (i.e. if a
     tool claims the presence/abuse of a specific security element, it
     is at least sometimes correct.)
   - the rate of false negatives is less than 100% (i.e. if an
     event occurs that is related to the presence/abuse of a specific
     security element, then sometimes the tool reports that event.)


**********************************************************
12. Revocation of CVE Compatibility
**********************************************************

1) If a repository has been approved for CVE compatibility, but at a
later time the Review Authority has evidence that not all requirements
are still being met, then the Review Authority MAY revoke its
approval.

2) Unless recommended by an Editorial Board member who does not have a
conflict of interest, the Review Authority SHOULD NOT perform a full
review of compatibility more often than once every three months.

3) The Review Authority MUST provide the repository owner and
Technical POC with a warning of revocation at least one month before
revocation is scheduled to occur.  The Review Authority MUST identify
the specific requirements that are being violated.

4) If the repository owner does not believe that the requirements are
being violated, then the owner MAY provide supporting evidence to the
Review Authority.

5) The Review Authority MAY delay the date of revocation.

6) If the owner's actions with respect to CVE compatibility
requirements have been found to be "intentionally misleading," then
revocation SHOULD last a minimum of one year.  The Review Authority
MAY interpret the preceding statement as it wishes.

7) If the approval is revoked, the repository owner MUST NOT apply for
a new review during the period of revocation.