RE: Sources: Full and Partial Coverage

[security curmudgeon <jericho@attrition.org>]

On Tue, 26 Jun 2012, Carsten Eiram wrote:

: > We're fairly ghetto, but OSVDB does a *lot* of source monitoring by hand.
: 
: It takes a fair amount of manual labour to do it properly. Naturally, we 
: don't sit in a browser visiting a huge list of sites every single day. 
: We have robots monitoring mailing lists and web sites, checking for new 
: discussions/content with certain keywords or new links.

Right. We have a weighted system based on the source, for priority in 
checking the source. ICS-CERT and Adobe are 'priority 1' for example, 
where low end software changelogs and bugtrackers are 'priority 9'. 
Regardless, we rely on a person looking at the sources.

: > : 5. Have set searches for phrases that indicate important vulnerabilities
: > : ("overflow", "XSS", etc).
: 
: That's one of the approaches we follow. Using that approach you, of 
: course, need a solid list of keywords to ensure proper coverage. If you 
: want to cover non-English sites you either need the same keywords in 
: those languages as well or first run the monitored sites through a 
: translation service e.g. Google Translate, hoping that it gets the 
: translation right to trigger the keyword matches. It's a solid way to 
: generate hits for further processing.

This is definitely a weakness for the automated parsing. Right now my 
parser is only good for English and French. The list of keywords I believe 
is robust. I had a solid list for several years, and then Steve Christey 
contributed his list which almost doubled my own. It generates a 
substantial amount of false positives, but I believe it is worth it as the 
false negatives are likely much smaller.