Thursday, April 30, 2009

Some days you just can't walk away.....

I apologize ahead of time for the marketing fluff in this post, I promise the next several posts after this will be much heavier on the tech and the cool. However, I just couldn't let this one go and neither could any of the Sourcefire VRT.

Today we got an anonymous email with the following pictures in them.




Now I know that whenever a marketing department makes a slide for a competitive package, it always makes them out to be the best. But this falls far from reality, these guys have Cisco over Sourcefire. Nothing wrong with Cisco, they make good routers and switches, but other than awesome comic book flash movies (therealm) and cool phones on 24 that's about it. They definitely didn't beat the VRT for 2008 MS Vulnerability coverage. (side note, there were 143 cve's from MS in 2008 not 140 - but who's counting?) (side side note there were 153 cve's issued total 10 were locals, and since we are a Network IPS we dropped those from the count)

Therefore I call these numbers into question, and I will now provide my own numbers, diligently researched by Alex Kirk (Sourcefire VRT, not marketing guy). This uses the same rules as provided in the first image above using grep and CVE references against available coverage data. Note: Only had TippingPoints and ours available so we didn't restate the other vendors numbers.





Additionally since the negative day response time thing is statistically silly at best, we calculated this number on our side by utilizing prior coverage detection. This included ms08-067 and MS08-052 which are detected by rules released in early 2006 or prior. It would be more statistically correct to give all prior coverage dates a value of 0, as negative numbers skew this data significantly. If we used this metric our response time would be .23 or essentially day 0 detection.

I now return you to your regularly scheduled blog of cool, and I will now return to playing with Adobe Javascript.......

int static_key_1 = 0x82056842;
Add to Technorati Favorites Digg! This

4 comments:

Master Ninja said...

I think you need to look at how actionable an alert is. So for MS08-067 you had older signatures, but most of these are high volume false positives already tuned out or otherwise ignored. Sure, if someone was doing anomaly detection on top of the old rules they might see it spike for a Conficker outbreak, but firewall port 445 activity spiked even more so I'd see the outbreak there before the IPS.

ISS didn't score well but their signatures didn't fire on false positives, so an older signature was actionable for MS08-067.

m3n1nja

extraexploit said...

the security poser and data&number crunch guys I think that are widespread worldwide. Usually they use numbers (and in the most of the cases wrong numbers)for hiding lacks in your technical skills. In my country a lot of people make money as security public relation guys or data crunch and hyper certified chain owner as well university degree to show in every speech or event or message post in the various ml. It's life.
I know only that when I'm in "debugging mode" I don't care to this guys.
Thank you for your information that you post here.

Best regards.

Martin said...

I'd like to expand on Master Ninja's comment because I agree. As a paying SF customer, I've been increasingly disappointed with signature quality. The new DCERPC2 preproc spews out just as many useless FP's on a busy Windows network with file servers as the others, making it almost entirely useless. (And yes, the sensor is tuned as much as you can tune for Windows traffic generating hosts.)

On a related note, I do not consider a vulnerability "covered" just because there is a rule matching a CLSID for a client-side app vuln. CLSID's are almost always obfuscated, so these sigs almost never fire, except for legit activity. This is because most VRT sigs have a content match anchor looking for the CLSID in clear text before the PCRE portion of the signature kicks in. I have seen an increased number of signatures implemented this way as well as older signatures, such as SID 4893. Writing them like this allows you to churn them out to claim coverage very quickly after the vuln is disclosed, but does little to actually protect your customers from real-world threats.

I have no idea if other IPS's write their signatures that way because they don't make them open, (which is why I don't buy other IPS's). Vuln research means nothing if the vulns you discover can't be detected with the signatures you sell.

extraexploit said...

Sorry for the mistake in my previous "romantic" comment where you have to replace "your technical skills" with "their technical skills". For the MS08-067 (win-smb-ntlm-code-executio) ISS generate a good numbers of false positive as well. It's the same for other "aggregated attacks" . Again,the same it's for HTTP related events that some time (a lot of times) are triggered but in most cases are false positive and are trigger only for some suspicious bad char within, for instance, URLs. Or what about fake malicious Javascript detected as "High" and usually is a friendly AD code or web user interaction related js code ? I'm not a ISS tuner but I know that these problem are very common in ISS users.
Sorry for the flame.