Re: Rule Load Formula

When they build cars they put them on a test track to see what kind of
MPG they get.  Obviously, real world MPG varies based on the road,
weather conditions, etc., but the numbers are still valuable for
comparing one car with another when purchasing.  Similarly, I think a
Snort proving ground pcap file could be created which is as regular as
the oval test track they use to test cars.  Ideally, it would
encompass every kind of traffic type that rules have to match on, but
it wouldn't have to have real-world content.  It would be more of a
way to map out the worst-case "mileage" of a rule.  Maybe it starts
with 100 valid full TCP connections in which the payload is a single
'A' and then 100 'AA', and then maybe a webpage with a header of 'A',
and so forth.  The idea is that you use this fake file to find the
worst-case matching scenarios and base your benchmark on that.  So
while real-world scenarios will always differ based on the traffic,
you could at least have a base measurement to go on.

Search algorithms themselves are measured by their worst-case
scenario, and so I don't see why a program comprised of search engines
can't be described in the same, finite way for worst-case.  Obviously,
your mileage may vary in the real world, but it provides everyone with
a starting point.  It is left to the individual administrator to know
their network and how their traffic will strain a given rule, but they
would be well served to have a precise and standardized way of making
comparisons on a large scale.

There are a lot of blanks to fill in here, but I think the basic idea
would be that for every method of search (pcre, content, etc.) there
is a worst-case scenario.  If a test rule file which contained the
gamut of searches was built to complement the worst-case corpus of
traffic (something like content:"A" when the packets contain one
hundred A's) then we could get good numbers on what rules cost without
having to worry about privacy concerns.

--Martin

On 1/12/07, Alex Kirk <alex.kirk-1iWHkkKxmXRZroRs9YW3xA@xxxxxxxxxxxxxxxx> wrote:
> >Something I've considered doing before and found to be more work that I
> >had resources to do, was to make an average traffic corpus and some kind
> >of framework to do the testing. Tools like tomahawk exist, but aren't
> >complete for this. There needs to be a very detailed statistical
> >analysis behind the testing.
> >
> >
> This would indeed be very interesting. Even if, in the end, you
> determined that the performance numbers you get out of this "average"
> traffic were marginally reliable at best, having a large body of sample
> traffic to test rules with would be exceedingly helpful in a lot of
> circumstances. I would think that you'd want constant additions to the
> pool of PCAPs, so that you get a better sample of the real world, and so
> that you don't fall into traps based on the type of traffic you once
> collected and have been using for some time.
>
> I think the biggest obstacle to something like this isn't even so much
> time as it is privacy. I could sit around and collect PCAPs on some of
> my personal and/or work boxes all day long, but I'd have to sift through
> them to remove private e-mail text, any cleartext-submitted passwords,
> etc. that I didn't want the rest of the universe to see. More
> imporantly, I'd feel like I needed the consent of everyone whose traffic
> I captured before publishing a PCAP. Got any thoughts on this angle of
> such a collection?
>
> Alex Kirk
> _______________________________________________
> Bleeding-sigs mailing list
> Bleeding-sigs-WwB1pFISwSkm7effSn6vN9HuzzzSOjJt@xxxxxxxxxxxxxxxx
> http://lists.bleedingthreats.net/cgi-bin/mailman/listinfo/bleeding-sigs