Re: 2.6.0 mem bug or me?

In 2.6.0 we use the ac method, it is the fastest, but does consume more 
memory and takes some initial resources to build the DFA it uses.  The 
acs/ac-banded/and ac-sparsebands/mwm/lowmem methods each use less 
memory, than the ac or ac-std methods.  However, we do not recommend mwm 
as it poses some DOS opportunities with repeated patterns. The low mem 
method is about 20% slower than the faster methods, but uses very little 
memory and very little initial resources.  Of couse you can also revert 
to the ac-std method that has been in use since 2.0 as well. It's 
startup is about 3x faster than the other ac methods.

Memory usage most to least is:

ac-std
ac
ac-banded
ac-sparsebands
mwm
acs
lowmem

startup processing most to least is

most
-----
ac
ac-banded
ac-sparsebands
acs

moderate
---------
ac-std

very little
---------
mwm
lowmem


0100 wrote:

>It seems that snort 2.6 uses a new algorithm for accelerating its
>matching performance. This dramatically has changed the daemon's
>performance profile.
>
>Take a look at:
>http://snort.org/docs/snort_htmanuals/htmanual_260/node10.html
>
>look for "search-method". They are now using the aho-corasick
>algorithm by default. This is a much faster matching than before due
>to it creating a finite state automata (FSA) in memory first before
>proceeding to do the match.
>
>So what does this mean in practical terms? The daemon's performance
>profile now is that it will run at 100% cpu for some time. The memory
>will continue to increase during this phase, as it is claiming memory
>and building the FSA based on the signature set you have loaded.
>Depending on your processor speed and signature set, this could take
>up to a few minutes. If, during this process, you run out of memory,
>the daemon will die. At the end of this phase you will see your CPU
>utilization crash dramatically down, and the memory usage will remain
>constant. Note how low your CPU utilization will stay after this
>process. Its actually quite astonishing compared with snort 2.4.
>
>Okay, so onto how to fix your issue. I have found the "acs" search
>method to be a good tradeoff for me. Put:
>
>config detection: search-method acs
>
>Into your snort.conf and try again. Hopefully this will work for you.
>If not, play around with some of the other search-methods in the docs.
>
>0100
>
>
>On 8/21/06, Andrew Jones <arjones@xxxxxxxxxxxxxxxxxxx> wrote:
>  
>
>>I assume there is something wrong with Snort, although i've heard
>>nothing about it. We have a machine with 2GB real memory and 1GB swap
>>(or maybe the other way around) that normally runs 7 Snort instances.
>>Under 2.4 we had no problems. with 2.6, some of the instances want up to
>>1,5GB memory. The Snort processes are constantly being killed. And to
>>any Snort developers who are reading, i set every single one of those
>>Snort instances to use the lowmem detection option. It doesn't help.
>>
>>                        -&
>>
>>Earl wrote:
>>    
>>
>>>-----BEGIN PGP SIGNED MESSAGE-----
>>>Hash: SHA1
>>>
>>>All,
>>>
>>>OpenSource snort 2.6.0
>>>OS: Fedora CORE3
>>>Mem: up to 1GB
>>>Swap: 512MB
>>>Problem: Upload latest VRT rules, restart snort (stop/start, not
>>>HUP)
>>> produces this error:
>>>Aug 21 15:07:51 localhost kernel: Out of Memory: Killed process
>>>##### (snort-plain)
>>>
>>>Although above appears in logs, it does actually *appear* to
>>>startup.  Is this a know snort issue, perhaps a FC ulimit tweak
>>>issue, or am I the only one seeing this?
>>>
>>>I've heard 2.6.1 is soon to be released.  Any idea when?
>>>
>>>Thanks.
>>>
>>>Earl
>>>-----BEGIN PGP SIGNATURE-----
>>>Note: This signature can be verified at https://www.hushtools.com/verify
>>>Version: Hush 2.5
>>>
>>>wkYEARECAAYFAkTp8soACgkQk7+e+4lPSm2FZgCgtzJWcRH0wlhkjuQGol/6C0eyIc8A
>>>nR7/kEGOrjhI3GptmBBQTGjJjOA4
>>>=QDu9
>>>-----END PGP SIGNATURE-----
>>>
>>>
>>>
>>>-------------------------------------------------------------------------
>>>Using Tomcat but need to do more? Need to support web services, security?
>>>Get stuff done quickly with pre-integrated technology to make your job easier
>>>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>>>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>>_______________________________________________
>>>Snort-devel mailing list
>>>Snort-devel@xxxxxxxxxxxxxxxxxxxxx
>>>https://lists.sourceforge.net/lists/listinfo/snort-devel
>>>      
>>>
>>--
>>GPG key / Schlüssel -- http://simultan.dyndns.org/~arjones/gpgkey.txt
>>Encrypt everything. / Alles verschlüsseln.
>>
>>
>>-------------------------------------------------------------------------
>>Using Tomcat but need to do more? Need to support web services, security?
>>Get stuff done quickly with pre-integrated technology to make your job easier
>>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>_______________________________________________
>>Snort-devel mailing list
>>Snort-devel@xxxxxxxxxxxxxxxxxxxxx
>>https://lists.sourceforge.net/lists/listinfo/snort-devel
>>
>>    
>>
>
>-------------------------------------------------------------------------
>Using Tomcat but need to do more? Need to support web services, security?
>Get stuff done quickly with pre-integrated technology to make your job easier
>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>_______________________________________________
>Snort-devel mailing list
>Snort-devel@xxxxxxxxxxxxxxxxxxxxx
>https://lists.sourceforge.net/lists/listinfo/snort-devel
>
>  
>


-- 
Marc Norton      Snort Team Lead
Sourcefire,Inc   410-423-1924
www.snort.org    www.sourcefire.com 


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642