Tuesday, April 17, 2012

Snort Performance and IP-Only Rules

One of the most frequent topics that comes up when I'm out speaking to customers, or when anyone from the VRT is discussing Snort on a mailing list, IRC channel, etc., is performance. Everyone wants to know how to make their rules faster - and many people are willing to go to extreme lengths to get the speed they want, devling into arcane corners of PCRE, looking for obscure Snort rule options, the works.

This wouldn't be notable, except for the fact that the people I've seen work the hardest at extreme performance tweaks in their rules are missing one of the most simple, fundamental things you can do to keep your Snort/Sourcefire boxes humming along: steering clear of IP-only rules. You know, rules that look like this:

alert ip 1.2.3.4 any -> any any (msg:"Evil stuff from IP 1.2.3.4"; classtype:bad-unknown;)

Ther problem with rules like that is that, without a static content match to supply to the fast pattern matcher, these rules will be evaluated by the main Snort engine on every packet that crosses the IDS. While each individual entry into the engine will be very brief - there's almost nothing to do - the process of entering the engine itself entails a non-zero amount of work, setting up structures, initializing memory, etc. If you're running hundreds, or even thousands, of IP-only rules (as we've observed at more than one production site in the field), the work necessary to enter the engine across potentially hundreds of millions of packets per second stacks up to the point that you end up chewing up all available system resources.

I understand why people were doing this in the past. There are plenty of groups out there who get or compile lists of known malicious IP addresses, and want to capture all activity related to those IPs; since many of those same groups have an IDS but not a full packet capture tool, they fell back to using Snort as that tool. Other groups simply wanted to ensure that their systems never visited these known malicious IPs, regardless of logging. Both groups of users spent years clamoring for a way to blacklist known-bad IP addresses in Snort; as far back as 2009, Marty Roesch was writing experimental code that provided that functionality. For many years, production systems had no way of dealing with intelligence that only provided IP addresses.

That all changed, though, with the release of Snort 2.9.1 in August of last year, however. That release included our IP reputation preprocessor, which checks IP addresses against a blacklist and/or whitelist at the time packet headers are decoded. Since this decoding is obviously already done across all packets crossing the engine, and since checking the obtained IP address against a properly formatted list in memory is exceptionally fast, this preprocessor allows you to have a list of tens of thousands or more IP addresses that you can check against with minimal performance overhead. (Note for Sourcefire customers: this feature is currently being beta tested on Sourcefire hardware.)

Many users seem to be unaware of this feature, despite it having been widely announced on the Snort Blog; others who are aware of it have been reticent to make use of it, for reasons I don't completely understand. In cases where we have seen the switch made, however, we've seen systems that were dropping up to 50% of their packets suddenly begin running smoothly again, processing everything that crossed them with ease.

So if you've got a sensor that's not up to speed, so to speak, do yourself a favor, and start with something easy - eliminating your IP-only rules in favor of the IP reputation preprocessor. You may be amazed at the difference it makes.
Add to Technorati Favorites Digg! This

4 comments:

jlewis said...

Info on the IP preprocessor is hard to come by. It would be super helpful if there was a heavily documented config example that someone could cut and paste and get running.

I managed to get the preprocessor running, after a lot of googling and trial and error. I've seen conflicting info on if rules are required.

The Walrus said...

Is the reputation preproc useful in an IDS (not IPS) setting? Does processing stop if an IP matches either list, or will processing continue?

Francis Turner said...

Possible dumb question here because I'm not seeing it answered in the docs.

How do you force snort to reload the files. I can see how this works in the shared environment but it isn't clear to me how it works in other simpler cases?

Matt said...

I think the comment by "The Walrus" is important. There are manay use cases of Snort, in particular - running as an IPS (or an IDS), but on a production network that owns the IDS etc. However what about the use case where a third party is providing monitoring, as a service for example? There is no capability to run inline, so as far as I can tell the IP Reputation preprocessor offers me nothing...

Many of our signatures are IP addresses of attacker infrastructure, where we want to inspect traffic to/from those IP's - from what I can tell, the reputation preprocessor doesn't cater for such a deployment so we are stuck with using traditional IP-Only rules.

What is your advice for use cases like what I have described here? I understand adding a content match will increase performance, however I want to detect anything to/from those IP's so it's a null point!

Thanks

Matt