RE: on the topic of CNA assignments... (was Re: Please welcome Kurt Seifried to the CVE Editorial Board)

["Boyle, Stephen V." <sboyle@mitre.org>]

Hi Brian,

 

Brian Martin said:

> I alluded to this in less-than-subtle ways in the last year. MITRE ignored

> the comments on the editorial board list.

 

We are not sure to which "this" you are referring, so we can't provide a specific, complete response. Please clarify your statement and MITRE will respond.

 

>  Last year, Will Dormann at CERT developed a tool to discover MitM issues

> in Android applications

> ...

> CERT, being a registered CNA, started issuing IDs for these vulns. About

> three to four weeks in, CVE/MITRE arbitrarily decided that Will should

> STOP assigning IDs for these vulns.

 

Mr. Dormann’s  research results were first published in the CERT Vulnerability Notes database (a CVE “Partial Coverage Source – Other”) and CERT was assigning CVE IDs to their findings in their role as a CVE CNA. MITRE became concerned about the sheer number of IDs to be assigned and discussed our concerns with CERT. After the discussion between MITRE and CERT (which included Mr. Dormann), MITRE and CERT agreed on a way forward, and CERT stopped issuing IDs based on the results of their automated testing of Android applications.

 

Key factors leading to the agreement between CERT and MITRE included:

• Disagreement between MITRE and CERT as to whether each of the findings noted by CERT represented a vulnerability, requiring further investigation by CERT before assigning CVE IDs.
• While MITRE agreed that there are valid vulnerabilities in a number of the findings cited, we did not accept the blanket assertion that there is a valid vulnerability in every identified Android app that does not verify a certificate.
• The CERT Vulnerability Notes Database is a partial coverage source and is a lower priority source.
• The sheer number of lower-priority results asserted to be vulnerabilities would require taking resources away from handling other, higher priority "must have" CVEs.
• The consideration that a relative explosion in the number of CVEs published would create problems for downstream consumers.
• The not-yet-completely-settled discussion of conditions by which CVE would handle results from automated testing.

 

> MITRE, decided on their own, without consulting the board, that a

> CNA should *STOP* issuing IDs to valid vulnerabilities. No valid reason

> was given, not to the public, and I bet a dollar not to Will himself. Just

> "oh god no stop it".

 

MITRE did not agree that all the CVE IDs being issued by CERT were for valid vulnerabilities. Our position was discussed with CERT at the time, and it was agreed that CERT would stop issuing CVE IDs for every finding from their research that had been published as a CERT:VU. This was noted in an email to the Board (2015-04-09), and was discussed at the Board meeting/conference call during RSA last year (2015-04-22).

 

MITRE often makes decisions on its own, but they are based on how we understand the Board wants CVE to be operated. MITRE viewed this specific case as a normal situation arising from a researcher’s results which needed to be clarified and discussed with the issuing CNA. MITRE views our decisions on coverage and prioritization simply as normal practice.

 

> Anyone on the board should be concerned right here. Why does MITRE have

> this absolute authority to stop issuing IDs on valid vulnerabilities?

 

MITRE does not have absolute authority. MITRE operates and manages CVE according to how we understand the Board wants us to operate it.

 

We hope that the above helps clear up our decisions regarding the automated testing of Android applications and answers the questions raised.

 

Best Regards,

The MITRE CVE Team

 

-----Original Message-----
From: owner-cve-editorial-board-list@lists.mitre.org [mailto:owner-cve-editorial-board-list@lists.mitre.org] On Behalf Of jericho
Sent: Saturday, November 07, 2015 2:03 AM
To: Art Manion <amanion@cert.org>
Cc: cve-editorial-board-list <cve-editorial-board-list@lists.mitre.org>
Subject: on the topic of CNA assignments... (was Re: Please welcome Kurt Seifried to the CVE Editorial Board)
Importance: High

 

On Wed, 4 Nov 2015, Art Manion wrote:

 

: On 2015-11-04 09:26, Kurt Seifried wrote:

:

: > One thing I do know is some of the external issues facing CVE (e.g.

: > people asking for CVE's on oss-sec that then take a while), I was

: > wondering if there are specific internal issues/challenges, e.g. are the

: > requests taking a long time/eating up resources because the requests are

: > poor? I often bounce private CVE requests back with "I need more info,

: > specifically X/Y/Z", and have been very firm with fuzzing reports (I

: > need minimized test cases and root cause analysis, not a pile of 100

: > text files that crash a command line app, and as it turns out that's all

: > the do).

:

: We (at CERT/CC) face similar issues in our CNA role.  We get researchers

: asking for a CVE ID for legitimate, but low severity/impact

: vulnerabilities that we otherwise would not handle or publish.  One

: option is to redirect the researcher to cve-assign, which sometimes

: comes back as "I already asked them and didn't get an answer so I'm

: asking CERT."

 

OK, so board... if the above is kind of foreign to you? Go read the

comics, ignore this.

 

For the 2% of you following board traffic, and my kind-of-subtle jabs in

this direction, grab your popcorn.

 

Kurt's mail, and Art's reply, are PERFECT. While MITRE can figure that out

for 'today', and moving forward, I will use this as an opportunity to

bring up the past. And the best part? I will pick on Art!

 

I alluded to this in less-than-subtle ways in the last year. MITRE ignored

the comments on the editorial board list. No one else on the board picked

up on it either, which is beyond discouraging. Because it means they are

likely oblivious to the real world of vuln disclosure.

 

So, OFFICIALLY.... MITRE... SPEAK TO THIS PLEASE.

 

Last year, Will Dormann at CERT developed a tool to discover MitM issues

in Android applications. He called it 'TAPIOCA', not to be confused with

the recent 'TAPIOCA' tool name. Because researchers are largely ignorant

and can't Google for shit.

 

CERT, being a registered CNA, started issuing IDs for these vulns. About

three to four weeks in, CVE/MITRE arbitrarily decided that Will should

STOP assigning IDs for these vulns.

 

Yep, process that for a minute or eight.

 

MITRE, decided on their own, without consulting the board, that a

CNA should *STOP* issuing IDs to valid vulnerabilities. No valid reason

was given, not to the public, and I bet a dollar not to Will himself. Just

"oh god no stop it".

 

Anyone on the board should be concerned right here. Why does MITRE have

this absolute authority to stop issuing IDs on valid vulnerabilities? You

can't argue they are valid or not, because MITRE actually spent the time

to write scripts to import Will's first run of vulnerabilities! CVE

auto-imported the data into the official CVE database, that feeds into

NVD, that is a cornerstone of our industry. In doing so, they missed the

many dozens of entries that had bad data due to the original import

scripting. To this day, we have CVE entries saying software is vulnerable,

with gibberish for the affected version number. Then, MITRE decided, no

more IDs... told Will, at CERT, which is a CNA, to stop assigning.

 

A year later, after alluding to it on the board, and MITRE ignoring

those comments... here we are. Still no explanation as to why that

decision was made, no hiring an extra intern at ~ 20k a year to import the

rest (on a budget of 1mil+)... basically, nothing holding them

accountable. Given MITRE/CVE's mission statement, that doesn't work for

me.

 

If you are wondering what all this means, please ask yourself why. Why

didn't you notice this last year? It was center-stage of the vuln

disclosure world, in many ways. No offense, but if you didn't notice me

bringing it up on list, and didn't notice it happening, are you really

suited to be on the board? 2014 represented a near 8x increase in vuln

disclosures. Yet, that isn't reflected in CVE at all, and it wasn't

brought to our attention, or our vote. Are you really comfortable with a

tax-payer funded VDB hiding ~ 80% of the disclosed vulnerabilities any

given year?

 

: I believe that CVE looks for a reasonably trusted public source of

: information (this might get to the sources and products lists) on which

: to base a CVE ID assignment and entry.  So a researcher publish

 

Yet, CERT is not trusted. See above. If we can't trust CERT to be a CNA...

who can we trust?

 

Certainly not Apple, who refuses to answer mails clarifying dupe CVE

assignments (Oct 2015).

 

Certainly not Microsoft or Adobe, who keep assigning 2015 IDs to issues

found and disclosed to them in 2014.

 

Certainly not IBM, who has released several hundred advisories using the

wrong CVE ID, that clearly states it is vendor-specific, and who has

been told by me and MITRE to stop...

 

Certainly not Cisco, who is cherry-picking which vulns get CVE

assignments, and when asked about public vuln information (that they

published) opt to redact information instead of clarifying it per

request...

 

Is there any wonder I have been asking for, and waiting for CNA guidelines

to be officially published?

 

The entire system is broken from the ground up. No one is policing it. The

rare times some asshole polices it externally, they get ignored over and

over.

 

: themselves or drop mail on full-disclosure might not be enough to

: support CVE ID assignment and writeup (in the case that neither CERT nor

: the vendor publish).

 

Oh god, stop there. F-D or Bugtraq are your 1999 or 2001 examples. This is

2015, you can't use either as an example to disclosure challenges. Look to

oss-sec first, and you will see why I am shocked I had to fight for three

years to get Kurt on board. Then consider the slightly easier sources like

EDB or PS... then for pure nostalgia, look to Bugtraq or FD, which are

front-and-center on the official sources MITRE monitors.

 

: While I support the idea that every vulnerability should have an

: identifier -- ideally a CVE ID -- there are tradeoffs with quality,

: quantity/scope/coverage, assignment speed, and resources.  It may be

: that working policy is that certain vulnerabilities just don't get CVE

: IDs?  Should a CNA (CERT) shunt requests to cve-assign or just say "no?"

 

If a CNA says "no" to requests in their own software, for ANY reason, they

should not be a CNA.

 

End of story.

 

A volunteer effort, run by 3 people in their spare time, doubled CVE's

output for half a decade. Each year, MITRE got 1mil+ from the

government, while these volunteers did it in their spare time.

 

You simply cannot argue about CVE and effectiveness, without addressing

that point. If MITRE's beauracracy is so convoluted and hindering to

the process, we need to consider alternatives.

 

.b