RE: Sources: Full and Partial Coverage

[Carsten Eiram <che@secunia.com>]

Joining the discussion quite late after five weeks vacation and a lot of catching up with my inbox...

Below are my comments to sources that should be fully covered with a couple of additions based on what I know large parts of our customer base specifically care about - some of these have very likely already been suggested by other members of the board:


> SHOULD BE FULLY COVERED
> -----------------------
> US-CERT: Technical Cyber Security Alerts 
> RealNetworks (real.com) 
> Apple
> EMC, as published through Bugtraq 
> VMware
> Google: Google Chrome (includes WebKit)
> IBM: issues in IBM ISS X-Force Database
> Internet Systems Consortium (ISC)
> MIT Kerberos
> Adobe
> Apache Software Foundation: Apache HTTP Server
> Cisco: Security Advisories/Responses
> HP: Security Bulletins
> Microsoft: Security Bulletins/Advisories 
> Mozilla
> Oracle

I agree to all of the above with the following additions:
* Symantec
* CA
* Citrix
* F5
* Blue Coat
* Juniper (access restricted)
* SAP (access restricted)
* Check Point
* IBM (AIX, Lotus) <-- both provide mailing lists and/or security pages for easy tracking compared to most stuff from IBM
* ICS-Cert
* ZDI + DVLabs [1] (also sent to lists and can be covered that way)
* iDefense VCP [1] (also sent to lists and can be covered that way)
* Exodus Intelligence EIP (not sure if they will be publishing to lists - if they are willing to buy a vulnerability report, we likely care about it)[1]
* Secunia Research (only some internally discovered vulnerabilities are sent to lists; no reports coordinated via SVCRP are currently sent to lists unless reporters do so themselves - note that Secunia is already a CNA and assigns CVEs ourselves so CVE team just has to update descriptions, references etc.)
* Kernel.org
* Novell
* PHP
* Kerberos / Heimdal

[1]: See other discussion about CNAs.


> SHOULD BE MONITORED BUT SELECTIVELY COVERED (being demoted)
> -------------------------------------------
> US-CERT: Vulnerability Notes [1]
> Symantec: SecurityFocus BugTraq (securityfocus.com/archive/1) [1]
> Symantec: SecurityFocus Bugtraq ID (securityfocus.com/bid) [1]
> Full Disclosure [1]
> OSVDB [1]
> SecurityTracker [1]
> FreeBSD [2]
> NetBSD [2]
> OpenBSD [2]
> Mandriva [2]
> oss-security [3]
> IBM: issues not in IBM ISS X-Force Database [4]

I agree that all of the above with the exception of Mandriva, which I see no need to cover at all, should be demoted to selective coverage.

---

I noticed that there has also been some discussion w.r.t. coverage of sources vs. coverage of products. I recommend that CVE continues focusing on source coverage instead of trying to fully cover any products for a number of reasons.

At the end of the day, when we are discussing sources, we are picking the ones, which we believe provide the best possible coverage of the majority of the products that our users care about. We are not picking sources at random so ultimately we end up achieving close to the same goal, but much "cheaper".

Defining a list of sources that should be covered (either fully or selectively) is easier to follow up on w.r.t determining whether or not coverage is as expected and then optionally adjust accordingly. Determining whether or not a product is fully covered is much harder as it requires monitoring more sources in a variety of languages with new frequently popping up.

When Secunia started out, we focused on sources and picked the major ones we felt provided the best coverage of the most popular products. We continuously added sources as interesting ones popped up. Handling this was pretty easy and provided an excellent, broad coverage. 

As we grew and got more customers, who specifically cared about certain products, we had to change our focus to become more product-oriented; now we had a commitment to those customers as an alerting service to provide advisories about all reports for the products they selected to receive alerts on (i.e. adding deep coverage instead of previously just broad coverage).

Today, we track an immense number of sources in order to provide as broad and deep a coverage as possible. I, however, estimate that an extremely small number of these sources likely provide us with 90-95% of the valid reports that we cover. The rest of the monitored sources (the vast majority) only provide the last 5-10% (where many are not for popular products), but I estimate that at the same time those sources provide ~80% of the noise we have to go through every single day. 

If you want to do full coverage of any popular product, you need to do it properly and that requires tracking a lot of very random sources and adding new ones regularly (including Chinese blogs where most of the time the quality of the local jaozi is discussed with one or two blogs a year about new vulnerabilities in Windows). I'm sure everyone on this board representing a VDB is familiar with this challenge.

We need to determine what we and others should expect of CVE. Personally, I don't see CVE as having the same commitment to users as alerting services like Secunia, SecurityFocus, X-Force etc. 

As a CVE consumer, I expect CVE to provide me with CVE identifiers for as many vulnerabilities as possible - the more prominent the source that has the information, the more important it is for me (and our customers) that it has a CVE identifier assigned. Secondly, I (and our customers) expect that from those selected sources, the most popular products are assigned CVE identifiers before phpGolf. 

I, therefore, find the source discussion most important - we can then always discuss which products/vendors should receive higher priority among those sources later if relevant even though we indirectly already have when agreeing on the sources.

Finally, since CVE is not competing with any VDBs, they can as far as I'm concerned rely quite a bit on VDBs to pick up vulnerabilities from random sources instead of doing it themselves. Also, if no VDBs or major sources cover a specific vulnerability report, how important is it then for it to have a CVE identifier assigned? If a critical vulnerability in a popular product, then the VDBs have failed, but will likely pick it up eventually (and CVE can then catch it from there) - I don't consider it to be the responsibility of the CVE team to uncover it. 


-- 

Med venlig hilsen / Kind regards


Carsten H. Eiram
Chief Security Specialist

Follow us on twitter
http://twitter.com/secunia
http://twitter.com/carsteneiram

Secunia
Mikado House
Rued Langgaards Vej 8
2300 Copenhagen S
Denmark

Phone   +45 7020 5144
Fax       +45 7020 5145


> -----Original Message-----
> From: owner-cve-editorial-board-list@lists.mitre.org [mailto:owner-cve-
> editorial-board-list@lists.mitre.org] On Behalf Of Mann, Dave
> Sent: 10. maj 2012 21:50
> To: cve-editorial-board-list
> Subject: RE: Sources: Full and Partial Coverage
> 
> Folks,
> 
> Three comments...
> 
> 1) Our language has moved from "must have/nice to have" to "fully
> covered/partially covered".
> 
> 2) In our current discussion, we are only considering sources that you all
> identified as "must haves" in our prior discussion.   The list that I posted last
> Friday broke your previous "must haves" into 2 sub-groups:  sources that the
> CVE team agrees should be "fully covered" and sources that the CVE team
> believes should be demoted to "partially covered status".
> 
> THE PRIMARY QUESTIONS WE'RE SEEKING GUIDANCE ON ARE:
> A) SHOULD ANY OF OUR SUGGESTED PARTIALLY COVERED SOURCES BE
> PROMOTED BACK TO FULLY COVERED STATUS?
> B) ARE THERE ANY OTHER SOURCES YOU BELIEVE SHOULD BE FULLY
> COVERED?
> 
> 3) As you consider these questions, please bear in mind that we have a very
> long list of sources previously designated as "nice to have".   We would ask
> that you hold your suggestions for other partially covered sources (aka nice
> to have) source for later when we consider the full list of partially covered
> sources (in addition to those we suggest demoting).
> 
> 
> 
> Here are the lists again, along with a list of sources that have been nominated
> as needing to be fully covered.  We would like more discussion on the fully
> covered sets.  Note, we may not be able to cover all of the sources being
> nominated as full coverage, so please consider and defend your nominations
> in that light.
> 
> 
> SHOULD BE FULLY COVERED
> -----------------------
> US-CERT: Technical Cyber Security Alerts RealNetworks (real.com) Apple
> EMC, as published through Bugtraq VMware
> Google: Google Chrome (includes WebKit)
> IBM: issues in IBM ISS X-Force Database
> Internet Systems Consortium (ISC)
> MIT Kerberos
> Adobe
> Apache Software Foundation: Apache HTTP Server
> Cisco: Security Advisories/Responses
> HP: Security Bulletins
> Microsoft: Security Bulletins/Advisories Mozilla
> Oracle
> 
> 
> SHOULD BE MONITORED BUT SELECTIVELY COVERED (being demoted)
> -------------------------------------------
> US-CERT: Vulnerability Notes [1]
> Symantec: SecurityFocus BugTraq (securityfocus.com/archive/1) [1]
> Symantec: SecurityFocus Bugtraq ID (securityfocus.com/bid) [1]
> Full Disclosure [1]
> OSVDB [1]
> SecurityTracker [1]
> FreeBSD [2]
> NetBSD [2]
> OpenBSD [2]
> Mandriva [2]
> oss-security [3]
> IBM: issues not in IBM ISS X-Force Database [4]
> 
> 
> PRESENT BIG CHALLENGES THAT MERIT DISCUSSION AT A LATER TIME
> ------------------------------------------------------------
> Debian
> Red Hat
> Attachmate: SUSE
> Ubuntu (Linux)
> 
> 
> 
> Requests for Additional Fully-Covered Sources
> ----------------------------------------------
> Juniper - JTAC Technical Bulletins
> Citrix / Xen
> ASF: Apache Tomcat
> Samba Security Updates and Information
> PHP
> FoxIt Support Center - Security Advisories Symantec Security (Not BIDs but
> actual Symantec Advisories) McAfee Security Exploit Database (for entries
> containing exploit code)
> 
> -Dave
> ==========================================================
> ========
> David Mann | Principal Infosec Scientist | The MITRE Corporation
> ------------------------------------------------------------------
> e-mail:damann@mitre.org | cell:781.424.6003
> ==========================================================
> ========