RE: [CD] CD Proposal: SF-LOC (Software flaws in different lines o f code)

[Bill Fithen <wlf@cert.org>]

On Thu, 15 Jun 2000, David LeBlanc wrote:

> > From: Bill Fithen [mailto:wlf@cert.org]
>
> > It seems truly fundamental to me that the nature of a vulnerability is
> > independent of the mechanism(s) that can be used to correct it. There
> > may be many ways to correct a vulnerability. In fact, it seems obvious
> > to me that a "list" of such corrections of countermeasures is an
> > entirely separate undertaking on a par with CVE (perhaps CCE?).
>
> This may seem to be fundamental, but practically it is more difficult than
> that.  For example, say I have a bad DLL that exposes 10 symptoms, but the
> cure is to update this one library.  I would say that the vulnerability is
> the bad library, but that there are numerous possible exploits as a result.
> The recent problem in the Linux kernel illustrates the issue.  So should we
> issue several dozen CVE entries about apps that su because of one flaw in
> certain Linux kernels?  Or should we issue one entry that says this one flaw
> causes a lot of problems? I'll go for one entry.

The problem here is one of perspective.

At one extreme is the software engineering security analyst who is
responsible for studying source code, understanding mistakes, and
transforming that knowledge into improved engineering practices. In
his mind, each minimal grouping of source code lines that collectively
creates a flaw is a vulnerability and it is imperative to record them
this way because each grouping represents an example of a practice
that should be avoided. And even groupings that have exactly the same
contents but occur in different places are important from a software
engineering metrics point of view (how else can you measure whether
you're getting better than to count every flaw, even duplicates?).
This person would say, in an arrogant tone of voice, that obviously
the example you give of 10 symptoms is 10 vulnerabilities (to the
extent that they are independent of one another).

At the other extreme is the system administrator whose job it is to
make sure that the correct files are installed and maintained on the
correct machines. From his viewpoint, a given file is either the
correct one (or the best available) or it is not. So, it is sufficient
for him to say, a file has a vulnerability. He does not care if it has
one or 17. All he needs to know is that the best available replacement
file is as good as he can get.

>From the beginning CVE has been oscillating between these extremes
because we have on the board members that are representative of these
two extremes and everything in between.

> > If the goal of this rule is to help us split or merge based on what
> > the vendor says about his own product and there is no other way to
> > obtain that information than to infer it through the vendor's action,
> > I say that it makes no difference whether the vulnerabilities in
> > question are split or merged. We have so little information as to make
> > it useless.
>
> We'll probably have to handle this on a case-by-case basis.  All the vendors
> are different. Recent Microsoft bulletins have been fairly explicit about
> whether a hotfix corrects related or unrelated issues, so it should be
> fairly easy to deal with our products. For the rest of it, YMMV - you'll see
> everything from very terse announcements with little to no information, all
> the way up to knowing which line of code failed in which module.

My hope is that the more mainstream the product and the more
significant the vulnerability, the easier we will find collecting the
necessary information.

> > I would rather have a rule that says that if the previous rules fail,
> > then we have to depend on the vendor of the product to say whether to
> > split or merge and just "live" with their response. We can never have
> > enough information (short of someone reverse engineering the affected
> > code) to make a meaningful split/merge decision.
>
> Again, I'd point out that perhaps "rules" is the wrong term to use and the
> wrong approach. "Rules" implies a strict and inflexible process and if you
> attempt to apply "rules" to a situation with a lot of noise, there will be a
> lot of failures and/or the rules will become so complex that we'll need
> lawyers to write and interpret them. Perhaps it is better to set forth goals
> and guidelines. We can then use our collective judgement in order to try and
> reach those goals on a case-by-case basis. It also gets us out of the
> business of attempting to chart a decision tree that handles all possible
> issues.

I agree. I use "rule" above in the natural language sense, not in the
formal logic sense. None of these CD's can have logically consistent,
universally applicable rules.

> > All other things being inconclusive with respect to the split/merge
> > question, if we arrive at this rule and the vendor refuses to answer
> > the split/merge question, then it seems to me the only conservative
> > approach is to split. If we incorrectly split, the only consequence is
> > information redundancy. If we incorrectly merge, the consequence is
> > information loss. Once we split based on the lack of information, the
> > (possibly redundant) entries are marked as 'this is all we know' and
> > left that way until we know more. If at some point in the future (even
> > after they might be voted into CVE), if someone learns enough to
> > "prove" to us they should have been merged, then we merge them.
>
> I'd agree with this - but a merge doesn't entail irrevocable information
> loss - we still have the original source reports, and we can still split
> something later if we really need to.  We will probably err in both ways.

I also agree with this. But, I wasn't thinking so much in terms of
irrevocable loss of information than I was thinking how an uninformed
user of CVE might interpret the absense of the supporting information
that we squirreled away against the day when we might need to
reconsider the merge. From our perspective, no matter how we represent
the resulting CVE entries, barring some catastrophe at MITRE, we will
always have the complete set of information we had originally to
reconsider their representation. But CVE users will not have the
benefit of that hidden information. For that reason, I favor a
mistaken split over a mistaken merge.

Bill