I understand your points, but what I'm arguing is that if a given rule
can have a theoretical processing cost, then each Snort admin has at
least a possibility of evaluating it without estimation. If you know
that your sensor sees lots of HTTP traffic, then you can take that
into account when you see that the dport is 80. But given the large
number of HTTP sigs, it would be very helpful to know that sig A with
its 3 pcre's and 2 content matches takes 37 oinks and sig B with one
content followed by one pcre takes 12 oinks.
For a more accurate idea of what that will do to overall system load,
multiply the oinks by the total packets your netflow/IPAudit stats
show destined for port 80 at given points in time. (You are logging
your flows to SQL, right?). Then subtract those packets that your BPF
drops before it hits the detection engine (or only multiply by the
flows seen after the Snort sensor if you aren't running in-line).
That analysis will give you a chronological report of when a given
rule was a strain on a system, and when it wasn't. Then you can start
down the road of doing things like pulling rules/adding rules
on-demand based on overall system performance, but you can do it more
intelligently than simply removing a given ruleset with cron at a
certain time, i.e. it can be extremely granular.
A truly complete solution someday would allow for automatic
load-shedding with a sig directive like "shed" to tell Snort that if
it needs to drop 200 oinks from port 80 sigs, it can shed this rule
(hey, Snort's already keeping track of how many flows are on what
ports, right). Right now it would be tough to implement that since
Snort would not have the ability to know how much it would help to
shed a given rule. But if it were known how much load could be saved
by dropping a rule, the internal algorithms would be fairly simple.
This may seem like overkill now, but think how many more rules we will
need to have in a year's time, and think of how few rules we will be
able to safely drop from our active rule sets by then. I think having
theoretical rule performance measurements marks can really help you
get the most bang for your buck from your hardware, and would really
open the door to a lot of improvements to the engine and its
management as a whole.
On 1/11/07, Matt Jonkman
<jonkman-WwB1pFISwSkm7effSn6vN9HuzzzSOjJt@xxxxxxxxxxxxxxxx> wrote:
Good points. But to measure load, you have to consider the traffic that
the sensor will see, as well as the rules.
If a high load rule never gets hit in the chain because no traffic
appears that's close to it, then it's not a high load rule. It just
makes the snort startup a few milliseconds longer. :)
One way could be to use the performance stats stats in recent snorts.
You could run a ruleset on average traffic in your environment, then add
the rule in question and go for a similar period on similar traffic and
measure the difference. That's the only real way I see to get stats that
mean something to a specific environment, but even that's subjective as
traffic will be slightly different in each period.
Something I've considered doing before and found to be more work that I
had resources to do, was to make an average traffic corpus and some kind
of framework to do the testing. Tools like tomahawk exist, but aren't
complete for this. There needs to be a very detailed statistical
analysis behind the testing.
But the key I think is the traffic corpus. Throwing packet generator
traffic at snort is 100% wasted effort. Snort needs full streams, real
conversations, and real attacks to be taxed. Companies that throw out
claims of gigabit speed with a packet generator are wasting your time.
It's absolutely meaningless.
For an overall testing 'standard', something we could all use to compare
ourselves and new products to, the corpus in use would have to have a
wide sampling of all types of traffic. Plus, it'll need valid attacks
interspersed, plus network anomaly crud (fragments, dupes, mangled
packets, vlan tags, etc).
THEN... you'd need to test on multiple architectures to find the
bottlenecks in each technology so that you're not saying snort has Xmb/s
limit with this ruleset when the bottleneck was actually the PCI bus, or
the FSB. You'd need to measure all aspects of system performance, and at
many different traffic levels, and then chart that all.
Or, you can throw the sig into production and see when packets start to
drop. :)
Matt
Martin Holste wrote:
> Yes, I've seen that site, but I was unsatisfied with having to rely on
> their results and with the lack of an API (for lack of a better term).
> It seems worth it to me to figure out exactly what it takes (perhaps
> down to count/size of memcpy's, etc?) for a rule op.
>
> Applications of this formula would be mainly in scripting to run
> historical reports on rule matches per system load. Or, more
> importantly for those of us who are always redlining our boxes,
> answering questions like "how many pps will I drop with the addition
> of this rule?" or "what is the rule capacity of this given setup?" I
> think the latter has been more of an issue lately, especially when you
> look at the exponential growth of the rule sets.
>
> Making truly informed decisions about the entirety of the rule set
> seems like a worthwhile endeavor.
>
> On 1/11/07, Matt Jonkman
<jonkman-WwB1pFISwSkm7effSn6vN9HuzzzSOjJt@xxxxxxxxxxxxxxxx> wrote:
>> Turbosnort.com is probably what you were thinking about. By the vigilant
>> minds guys.
>>
>> Load has to consider traffic patterns, which this doesn't. Although it
>> is *some* measure of performance. I use them here and there when unsure.
>>
>> Matt
>>
>> Gentoo-Wally wrote:
>> > My math is not so good so... j/k
>> >
>> > Isn't there a site somewhere where you can copy a rule into it and it
>> > would test it for performance?
>> >
>> > Wally
>> >
>> > On 1/11/07, Martin Holste
<martin.holste-Re5JQEeQqe8AvxtiuMwx3w@xxxxxxxxxxxxxxxx> wrote:
>> >> It seems to me that it should be possible to create a somewhat simple
>> >> (or at least straightforward) formula for how much effort it takes to
>> >> process a given rule. It could be as simple as a unit of snort
>> >> processing so that rule comparisons can be made, though not translated
>> >> directly to actual CPU cycles or other measurable real-world events.
>> >>
>> >> I'm not sure if benchmarking or code-analysis (or some combination)
>> >> would prove the most accurate. Obviously, different
>> >> compiling/architecture options affect Snort's performance, but it
>> >> seems to me that a theoretical performance formula is tangible based
>> >> on the source code. I also think that it would be fair to assume that
>> >> the ac-bnfa matching algorithm is used, since it is destined to be the
>> >> default.
>> >>
>> >> So, to that end, do you think this mailling list could put together
>> >> something to start a list of values where an oink = 1 unit of Snort
>> >> processing load. Even estimated values would be helpful.
>> >>
>> >> Matching Oinks
>> >> content x (per char)
>> >> pcre x (per char)
>> >> rawbytes x (per char)
>> >> uricontent x (per char)
>> >> isdataat x
>> >> byte_test x
>> >> byte_jump x
>> >> ftpbounce x
>> >> regex x (per char)
>> >>
>> >> Modifiers Oinks
>> >> nocase x
>> >> depth x
>> >> offset x
>> >> distance x
>> >> within x
>> >>
>> >> Non-payload Oinks
>> >> fragoffset x
>> >> ttl x
>> >> tos x
>> >> id x
>> >> ipopts x
>> >> fragbits x
>> >> dsize x
>> >> flags x
>> >> flow x
>> >> flowbits x
>> >> seq x
>> >> ack x
>> >> window x
>> >> itype x
>> >> icode x
>> >> icmp_id x
>> >> icmp_seq x
>> >> rpc x
>> >> ip_proto x
>> >> sameip x
>> >>
>> >> We'd then need to figure out multipliers for single/all ports and IP
>> >> filters as well as other miscellaneous additions to the formula.
>> >>
>> >> So, if something like this has been done, please point me to it, but I
>> >> was unable to Google anything like this, and I figured that with all
>> >> of the seasoned Snort vets on this list we ought to be able to come up
>> >> with something.
>> >>
>> >> --Martin
>> >> _______________________________________________
>> >> Bleeding-sigs mailing list
>> >> Bleeding-sigs-WwB1pFISwSkm7effSn6vN9HuzzzSOjJt@xxxxxxxxxxxxxxxx
>> >>
>> http://lists.bleedingthreats.net/cgi-bin/mailman/listinfo/bleeding-sigs
>> >>
>> > _______________________________________________
>> > Bleeding-sigs mailing list
>> > Bleeding-sigs-WwB1pFISwSkm7effSn6vN9HuzzzSOjJt@xxxxxxxxxxxxxxxx
>> > http://lists.bleedingthreats.net/cgi-bin/mailman/listinfo/bleeding-sigs
>>
>> --
>> --------------------------------------------
>> Matthew Jonkman
>> Bleeding Edge Threats
>> 765-429-0398
>> 765-807-3060 fax
>> http://www.bleedingthreats.net
>> --------------------------------------------
>>
>> PGP: http://www.bleedingthreats.com/mattjonkman.asc
>>
>>
>> _______________________________________________
>> Bleeding-sigs mailing list
>> Bleeding-sigs-WwB1pFISwSkm7effSn6vN9HuzzzSOjJt@xxxxxxxxxxxxxxxx
>> http://lists.bleedingthreats.net/cgi-bin/mailman/listinfo/bleeding-sigs
>>
> _______________________________________________
> Bleeding-sigs mailing list
> Bleeding-sigs-WwB1pFISwSkm7effSn6vN9HuzzzSOjJt@xxxxxxxxxxxxxxxx
> http://lists.bleedingthreats.net/cgi-bin/mailman/listinfo/bleeding-sigs
--
--------------------------------------------
Matthew Jonkman
Bleeding Edge Threats
765-429-0398
765-807-3060 fax
http://www.bleedingthreats.net
--------------------------------------------
PGP: http://www.bleedingthreats.com/mattjonkman.asc
_______________________________________________
Bleeding-sigs mailing list
Bleeding-sigs-WwB1pFISwSkm7effSn6vN9HuzzzSOjJt@xxxxxxxxxxxxxxxx
http://lists.bleedingthreats.net/cgi-bin/mailman/listinfo/bleeding-sigs
| 
