RE: Sources: Full and Partial Coverage

[Carsten Eiram <che@secunia.com>]

> -----Original Message-----
> From: owner-cve-editorial-board-list@lists.mitre.org [mailto:owner-cve-
> editorial-board-list@lists.mitre.org] On Behalf Of security curmudgeon
> Sent: 25. juni 2012 21:02
> To: Art Manion
> Cc: 'cve-editorial-board-list'
> Subject: Re: Sources: Full and Partial Coverage
> 
> On Mon, 25 Jun 2012, Art Manion wrote:
> 
> : Do we really need to restrict the list of sources too heavily?  I'll
> : guess that Secunia and other places doesn't do all this monitoring by
> : hand...?
> 
> We're fairly ghetto, but OSVDB does a *lot* of source monitoring by hand.

It takes a fair amount of manual labour to do it properly. Naturally, we don't sit in a browser visiting a huge list of sites every single day. We have robots monitoring mailing lists and web sites, checking for new discussions/content with certain keywords or new links. 

However, that generates _a lot_ of hits, which educated people have to weed through by hand - that can't be handled automatically to a satisfactory degree (at least we haven't figured it out yet and we've been trying hard to automate as much as possible since it would save a lot of time).

We take two manual passes over the hits generated by the robots: First weeding out the obvious noise (fairly fast) and then assigning the rest for further processing where the not so obvious noise is weeded out (not so fast). The remainder is verified by the team and written up as advisories if valid and not dupes (definitely not fast).

However, as I pointed out in a previous mail, with a fairly small number of sources, you can get a very solid coverage of 90-95%. The lists of sources we're currently discussing here should definitely cover most and as I also suggested: Keep an eye on the VDBs for the rest; the CVE team is already monitoring these anyway to add references to the CVE entries.


> : 5. Have set searches for phrases that indicate important vulnerabilities
> : ("overflow", "XSS", etc).

That's one of the approaches we follow. Using that approach you, of course, need a solid list of keywords to ensure proper coverage. If you want to cover non-English sites you either need the same keywords in those languages as well or first run the monitored sites through a translation service e.g. Google Translate, hoping that it gets the translation right to trigger the keyword matches. It's a solid way to generate hits for further processing.

/Carsten