Re: dedup (was: definition)

On 1/11/07, Matt Jonkman 
<jonkman-WwB1pFISwSkm7effSn6vN9HuzzzSOjJt@xxxxxxxxxxxxxxxx> wrote:
> Gentoo-Wally wrote:
> > Sorry about the repost but I did not want this to get lost in the $hit
> > storm I apparently created :(...
>
> Nice job Wally. :)

Guess I'm a bad influence :)

> >
> >> From my experience the most difficult (or at least the most daunting)
> > aspect of tunning is not preprocessor tunning but rather weeding out
> > signatures that have nothing to do with my environments and ensuring I
> > have enabled all the relevant rules. I have usually found that my
> > sensors monitoring the internet traffic of a medium sized, mixed OS,
> > well maintained server farm really only needs around 2-4k of the 11k
> > available rules. I usually focus on attacks that I could actually be
> > vulnerable to. Some people subscribe to the idea that they want to
> > know of any potential attack regardless of it's actual penetration
> > potential. More power to them, but in a production environment, I'm
> > just not one of those people. I don't care about every attempt by a
> > script-kiddies trying to exploit the old IIS Unicode vuln to get
> > cmd.exe against my 50 server LAMP farm. If I want to know about stuff
> > that does not really affect me, perhaps for early warning or just
> > statistical analysis, I'll throw up another sensor to do that.
> >
>
> That's a good discussion and a good point. I personally subscribe to the
> "run every accurate and reliable rule your sensors can handle" camp.
> Why? Because it's very rare I setup an install that's not able to block.
>
> Why the difference? If you're not blocking, then you don't need to know
> about things you're not vulnerable to. Even if you're watching the
> console every second of the day, you can't respond fast enough to
> manually block an attacker that knows their stuff.
>
> If you ARE blocking, the automated blocking on reliable rules can
> instantly knock out someone with hostile INTENT. Not successful
> hostility, but intent. If your intent is not to browse a protected
> website, but to fiddle with php, then we don't want ya.
>
> There are many reasons to go either way, but that's the main difference
> I see. The number of benerated alerts will be similar, since most of the
> crud from an attacker and script kiddie will be killed after the first hit.

Good points! Most of my work is in the realm of IDS and not IPS so I
had not really looked at those issues from that perspective. THX!

> > Sorry got a little OT there...With that said, I've been doing this for
> > quite a while and even for me in a multi sensor IDS environment rule
> > maintenance and tunning can sometimes be difficult. Having had Jr. IDS
> > admins under me I know they hate rule maintenance and it can eat into
> > a lot of their time. Time I feel could be better spent reviewing
> > alerts. (yea, yea I'm a slave driver) I think that having 3
> > distributions of rules and no de-duplication process will only add to
> > the overhead of rule maintenance of already over worked IDS admins.
> > Every minute lost to tuning and maintenance is time that is taken away
> > from actual alert analysis, especially in underfunded and undermanned
> > sectors such as higher ed.
>
> AGREED!! We need a de-duping process. But there are many many issues,
> and a lot of work to be done to make that happen.
>
> >
> > Since it looks kind of like the OSSRC didn't get going maybe there is
> > something we could do? Maybe a dedup project? Say that the following
> > sid's were duplicates of the same rule with minor content differences
> > (hex vs. plain text or slight pcre diffs)...
> >
> > VRT
> > 1234
> >
> > Community
> > 4567
> >
> > BE
> > 7890
> >
> > I could imagine a list with an entry like...
> >
> > V1234,C4567,B7890
> >
>
> That was proposed. It ought to be pretty easily implementable in
> oinkmaster and other rule managers. You just tell it which ruleset you
> prefer and it enables the coverages from that, disables the other.
>
> The issues that came up in the initial discussions (didn't kill it, but
> were issues) were like:

As I was reading your questions below it became apparent to me that we
might have a different definition of duplicate signatures. Then I
realized it wasn't that our definitions differ but rather that there
are really 2 types of duplicates.

Type 1. Multiple rules looking for the exact same traffic but in different ways.

Ex.   One uses something like content:"blah"; and the other rule uses
content:"|62 6c 61 68|";

Type 2. Multiple rules that attempt to find the same issue, but in
different ways.

Ex.   One rule that is very accurate but computationally expensive and
a second that is less accurate but is less expensive.

I would say that Type 1 dups need not exist between _cooperative_ rule
distributions because they accomplish the same thing and have a
negligible performance difference.

With Type 2 dups both rules do need to exist because although they
look for the same issue they have different goals with relation to
accuracy vs. performance and perhaps other things.

With that said...

> 1. Who decides which is better? Is performance losses for total accuracy
> worth it, or are a few falses ok for a huge performance gain?  (needs to
> be a local decision)

If it is a Type 1 dup that call could be based on things like rule
readability. It is easier to read a rule with a plain text content
than a hex content.

If it is a Type 2 dup we do not actually want to remove one or the
other but rather document their existence and conditions in an a way
that can be parsed and used easily.

Expanding on my earlier format say the VRT rule is more accurate than
the Community and Bleeding rule but more resource intensive than both
also and the Bleeding rule is more accurate than the Community and
more more resource intensive. We could define these conditions like
this...

99V1234,53C4567,65B7890

Where the first position represents accuracy and the second represents
processing expense.

The important piece is identifying them. Right now if there exists a
rule in each of the VRT, Community, and Bleeding distributions that
fits a scenario similar to the one above it is likely that many users
using all three rule distributions probably have all three rules
enabled (which is probably counter productive) because they do not
know the duplication even exists.

> 2. What about the folks that don't pay for VRT?

What about them? If a sid is missing then the determination is between
the other two (or more) and does not affect the documentation of dups.

> How many other rulesets do we include?

How many others are there? I would say start with VRT, Community and
Bleeding since there is a fair amount of cooperation between us
already. Once established, maybe come up with some kind of requirement
to be included in the fun and games.

> 3. Who's going to do this work of reviewing EVERY rule, and EVERY
> revision?  Just maintaining the rules is enough to do. This groups that
> reviews would have to research every vuln to be able to make an educated
> decision as to which will be best...

And there's the rub. The biggest work load would be getting started
and documenting all the current dups. Once that was accomplished it
would be fairly easy to monitor new rules for duplication especially
if there was some type of group like the OSSRC doing something like
sid registration. I would almost think this would be a branch or
project of the OSSRC, especially once the initial grunt work was done.

Wally

> >
> > That is just an off the top of my head idea. Might be better was to do
> > this. Maybe I'm just crazy. Anyone think this whole idea is over kill?
>
> I'm definitely interested in the discussion, and hope something like
> that becomes a priority. If there are folks that are interested in doing
> the work (it'll be a LOT of work, especially to get it defined and
> started) then I'll chip in what I can as well as my support, and my
> support via my seat on the ossrc.
>
> Matt
>
>
> --------------------------------------------
> Matthew Jonkman
> Bleeding Edge Threats
> 765-429-0398
> 765-807-3060 fax
> http://www.bleedingthreats.net
> --------------------------------------------
>
> PGP: http://www.bleedingthreats.com/mattjonkman.asc
>
>
> _______________________________________________
> Bleeding-sigs mailing list
> Bleeding-sigs-WwB1pFISwSkm7effSn6vN9HuzzzSOjJt@xxxxxxxxxxxxxxxx
> http://lists.bleedingthreats.net/cgi-bin/mailman/listinfo/bleeding-sigs