[CVEPRI] Updated CVE Compatibility Requirements document

["Steven M. Christey" <coley@linus.mitre.org>]

All,

Below is an updated requirements document for CVE compatibility.  We
will discuss these requirements during a portion of the Editorial
Board meeting, then finalize the initial requirements document shortly
afterward.  The requirements will be posted on the new CVE web site.

General change summary:

- CVE searchability requirement slightly weakened for security tools

- Some Good Faith requirements upgraded from SHOULD to MUST

- Accuracy requirements simplified and clarified


- Steve



Requirements and Recommendations for CVE Compatibility
------------------------------------------------------
Steve Christey
The MITRE Corporation
coley@mitre.org
CVE web site: http://cve.mitre.org

Document version: 0.4
Date: August 9, 2000


*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***
*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***
*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***

This is a draft report and does not represent an official position of
the MITRE Corporation.  (c) 2000, The MITRE Corporation.  All rights
reserved.  Permission is granted to redistribute this document if this
paragraph is not removed.  This draft is subject to change without
notice.

*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***
*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***
*** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT *** DRAFT ***


*****************************
Table of Contents
*****************************
1. Definitions
2. Prerequisites
3. High-Level Requirements
4. Requirements with respect to CVE Versions
5. Requirements for Repositories that Use Candidate Numbers
6. Additional Requirements for Web Sites
7. Additional Requirements for Security Tools
8. Additional Requirements for Other Repositories
9. Accuracy Requirements
10. Good Faith Requirements
11. Revocation of CVE Compatibility


*****************************
1. Definitions
*****************************

Security Element - a database record, email message, security
advisory, assessment probe, signature, etc., which is related to
information system security information.

Repository - a collection of security elements, e.g. a database,
security tool, or web site

Owner - the owner or maintainer of the repository

Map/Mapping - the specification of relationships between security
elements in a repository and the CVE entries or candidates that are
related to those elements.

Review - the process of determining whether a repository is CVE
compatible.

Review Version - the version of CVE that is being used for determining
compatibility of a repository.

Review Authority - an entity that performs a Review.  MITRE is the
only Review Authority at this time.  Currently, there is no process
for assigning other Review Authorities.


**********************************************************
2. Prerequisites
**********************************************************

1) The repository owner MUST be a valid legal entity, e.g. a
corporation or a specific individual.

2) The Owner MUST provide MITRE with free access to the repository.
This allows MITRE to (a) review the mapping for accuracy, and (b)
identify repository elements that must be added to CVE.

3) The repository MUST provide additional value or information beyond
that which is provided in CVE itself (i.e. name, description,
references, and associated candidate data).


**********************************************************
3. High-Level Requirements
**********************************************************

These are the high-level requirements for all repositories.  They are
described in detail in later sections.

1) If the repository does not satisfy all requirements, then it MUST
NOT be advertised that it is CVE-compatible.

2) The repository MUST allow users to locate security elements using
CVE names ("CVE-Searchable").

3) When the repository presents security elements to the user, it MUST
allow the user to obtain the asociated CVE names ("CVE-Reportable" or
"CVE-Output").

4) The repository MUST properly identify the CVE Review Version that
the Review Authority has used for determining compatibility.

5) The repository MUST satisfy the Accuracy requirements, as specified
below.

6) The repository MUST satisfy the Good Faith requirements, as
specified below.

7) The repository MUST include a prominent reference to the CVE web
site (http://cve.mitre.org)


The repository is NOT REQUIRED to do any of the following:
  - use the same descriptions or references as CVE
  - use CVE candidate numbers
  - include every CVE entry in its repository


*********************************************************************
4. Requirements with respect to CVE Versions
*********************************************************************

This set of requirements poses some challenges.  On one hand, it is
important that end users know how "up-to-date" a repository is with
respect to its mappings, and the CVE version can play a role in this.
However, it is not yet defined how Review Versions and Reference
Versions should be assigned.  As CVE versions change on a regular
basis, it must be considered whether CVE-Compatibility should be
re-evaluated for each version, or if it should be regarded more as a
"process" during which a full-fledged review is only performed
periodically.

1) The repository MUST be reviewed for CVE compatibility with respect
to a specific CVE version, i.e. the Review Version.

2) The repository MUST clearly identify which Review Version was used
to determine compatibility.

3) The repository SHOULD be compatible with a CVE version that is an
official CVE "Reference Version" as announced by MITRE.  On a periodic
basis (approximately once per quarter), MITRE and/or the CVE Editorial
Board will announce that a specific CVE version is to be used as a
Reference Version.

4) If the Owner believes that the repository would satisfy
compatibility requirements for CVE versions that are more recent than
the Review Version, then the Owner MAY report that it is "up-to-date"
with respect to the more recent version.

5) If the repository uses candidates, then it SHOULD indicate the
release date associated with the candidate information.


*********************************************************************
5. Requirements for Repositories that Use Candidate Numbers
*********************************************************************

A repository MAY include CVE candidate numbers.  It is not required to
do so.  CVE Compatibility is only established with respect to the
entries on the official CVE list.  Candidates are not expected to
become a part of the requirements for CVE compatibility.  In cases
where repositories use candidates, however, certain recommendations
must be followed.

If the repository uses candidate numbers, then:

1) The repository MUST clearly indicate to the user that the candidate
number is not an accepted CVE entry.  The use of the CAN- prefix
itself is sufficient to indicate candidate status.

2) The repository SHOULD include additional supporting text that
describes how candidates are different than official entries.

3) The repository SHOULD include the text "(under review)" after the
candidate number when it is used in natural language text
descriptions.

4) If a candidate has become an official CVE entry within the Review
Version (or earlier), then the repository SHOULD replace that
candidate number with the associated CVE number/numbers.  In other
words, the owner must keep up-to-date with respect to candidates that
become official entries.

5) If a user performs a search using YYYY-NNNN, the repository SHOULD
return the security elements that correspond to CAN-YYYY-NNNN or
CVE-YYYY-NNNN.

6) If the repository contains the official entry CVE-YYYY-NNNN, but
the user searches using CAN-YYYY-NNNN, then the repository SHOULD
return CVE-YYYY-NNNN, and vice versa.


**********************************************************
6. Additional Requirements for Web Sites
**********************************************************

1) The web site MUST allow a user to type in a CVE name and retrieve
the related security elements from the repository ("CVE-Searchable").
The web site MAY satisfy this requirement by providing a web page
which links CVE numbers to the related elements.

2) A web page that provides details for an individual security element
MUST identify the CVE entry/entries that map to that element
("CVE-Output").  If web pages are not provided for individual
elements, then the CVE name MUST be associated with the security
element in some other fashion.

3) A URL "template" SHOULD be provided which allows a computer program
to easily construct a link that accesses the search capability as
outlined in (1).  This will allow CVE-compatible web sites to easily
link to each other, and reduce the effort of link maintenance.
Examples:

  http://www.example.com/cve/my-db-cve-name.cgi?name=CVE-YYYY-NNNN
  http://www.example.com/cve/CVE-YYYY-NNNN.html

If the URL template is for a CGI program, the program MUST accept the
HTTP "GET" method.

4) The web site SHOULD use approved "generic" entries in the same way
that normal CVE names are.

5) The web site MAY link to specific CVE entries on the official CVE
web site.  The URL template is:

   http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-YYYY-NNNN


**********************************************************
7. Additional Requirements for Security Tools
**********************************************************

A security tool is a software application or device that examines a
host or network and produces information that is related to
vulnerabilities or exposures, e.g. an assessment tool, intrusion
detection system, or a risk management product.  A "task" is a
security tool's probe, check, signature, etc. which performs some
action that produces security information.

1) For any report which provides details for individual
vulnerabilities or exposures, the tool MUST allow the user to include
CVE names in that report ("CVE-Searchable").

2) The tool MUST allow the user to use CVE names to locate associated
tasks in that tool ("CVE-Output").  The tool MAY satisfy this
requirement by providing the user with a mapping between task names
and CVE names.

3) The tool SHOULD provide the user with a list of all CVE entries
that are associated with the tool's tasks.

4) The tool SHOULD allow the user to specify a set of tasks by
inputting a file of CVE names.

5) The interface of the tool SHOULD allow the user to browse, select,
and deselect a set of tasks using individual CVE names.

6) If the tool does not have a task that is associated with a CVE name
as specified by the user in (4) or (5) above, then the tool SHOULD
notify the user that it cannot perform the associated task.


**********************************************************
8. Additional Requirements for Other Repositories
**********************************************************

Requirement (1) is an attempt to characterize what "CVE-Searchable"
and "CVE-Output" might mean for other repositories besides web sites
and tools.  Requirement (2) is less well-defined because it is unclear
as to what CVE-Compatibility may mean in less "traditional"
repositories.

1) If a GUI is provided with the repository, then:
   - A search capability MUST be provided that allows a user to use a
     CVE name and retrieve the related security elements from the
     repository.
   - A window/dialog that provides details for an individual security
     element MUST allow the user to identify the CVE entry/entries
     that map to that element.
   - A user SHOULD NOT be allowed to change the map between a security
     element and the related CVE entry/entries.

2) If a GUI is NOT provided with the repository, then the repository
SHOULD be structured so that:
     - A query or other programmed search mechanism can locate
       related security elements when provided with a CVE name.
     - The CVE name(s) for a specific security element can be accessed
       from each element using a query or other programmed search
       mechanism.
     - If a repository provides a mapping to the end user, then it
       is regarded as satisfying these two requirements.


**********************************************************
9. Accuracy Requirements
**********************************************************

A Review Authority is responsible for reviewing the accuracy of a
repository.  CVE Compatibility only facilitates data sharing when the
mappings are accurate.


Review Authority Requirements
-----------------------------

1) The Review Authority MUST define a Sample Size, which is the
percentage of security elements and/or the number of elements to be
examined by the Review Authority.  The Review Authority SHOULD use a
minimum Sample Size of either 50 elements or 20% of the repository.
The Review Authority MAY review every element in the repository.

2) The Review Authority MUST publicize the sampling method.  The
Review Authority MAY use non-random sampling techniques.  The
resulting sample of the repository is referred to as the Review
Sample.

3) The Review Authority MUST define a Minimum Accuracy Percentage,
which is the percentage of security elements in the Review Sample that
reference CVE identifiers, i.e.:

   - the CVE name/names of related entry/entries
   - a GENERIC specifier which is approved for use, except
     GENERIC-UNMAPPED.
   - a blank field which is equivalent to GENERIC-MAP-NOMATCH.

4) The Minimum Accuracy Percentage SHOULD be 80% or higher.

5) The Review Authority MUST use the same sampling method for all
repositories that are being reviewed with respect to a particular
Review Version.


Repository Requirements
-----------------------

1) The repository owner MUST provide the Review Authority with free
access to the repository for review.

2) The repository MUST meet or exceed the Minimum Accuracy Percentage.



**********************************************************
10. Good Faith Requirements
**********************************************************

Good Faith on the part of the repository owner is essential,
especially while CVE Compatibility is still being refined as a
concept.

1) The repository owner MUST provide a technical POC who is qualified
to answer questions related to the accuracy, maintenance, or
construction of the mapping.  The POC SHOULD be made available to the
Review Authority and to the repository owner's users.

2) The repository owner SHOULD correct errors in mappings in a timely
fashion, i.e. within one month or the repository's average "update
cycle" in the past year, whichever is greater.  (Example: a tool that
provides updates every 3 months would be expected to correct a mapping
error within 3 months of the notification of the error).

3) The repository owner SHOULD prepare and sign a statement that, to
the best of their knowledge, the owner has not provided any false
information in the mapping.


Security tools also have the following requirement.

4) The tool vendor SHOULD sign a statement which warrants that, for
each task that is associated with a CVE entry, all of the following
are true:
   - the rate of false positives is less than 100% (i.e. if a
     tool claims the presence/abuse of a specific security element, it
     is at least sometimes correct.)
   - the rate of false negatives is less than 100% (i.e. if an
     event occurs that is related to the presence/abuse of a specific
     security element, then sometimes the tool reports that event.)


**********************************************************
11. Revocation of CVE Compatibility
**********************************************************

1) If a repository has been approved for CVE compatibility, but at a
later time the Review Authority has evidence that not all requirements
are still being met, then the Review Authority MAY revoke its
approval.

2) Unless recommended by an Editorial Board member who does not have a
conflict of interest, the Review Authority SHOULD NOT perform a full
review of compatibility more often than once every three months.

3) The Review Authority MUST provide the repository owner and
Technical POC with a warning of revocation at least one month before
revocation is scheduled to occur.  The Review Authority MUST identify
the specific requirements that are being violated.

4) If the repository owner does not believe that the requirements are
being violated, then the owner MAY provide supporting evidence to the
Review Authority.

5) The Review Authority MAY delay the date of revocation.

6) If the owner's actions with respect to CVE compatibility
requirements have been found to be "intentionally misleading," then
revocation SHOULD last a minimum of one year.  The Review Authority
MAY interpret the preceding statement as it wishes.

7) If the approval is revoked, the repository owner MUST NOT apply for
a new review during the period of revocation.