Select Page


Five Vendors Achieve Recommend Rating, but Wide Ranges in Effectiveness, Performance and Value Prove that Buyers Should Carefully Review Products before Purchasing

CARLSBAD, Calif., January 10, 2011 – NSS Labs, Inc., the leading independent security testing organization, today announced the release of its latest Network Intrusion Prevention System (IPS) Comparative Group Test Report for the fourth quarter of 2010.

Key findings from the report show:

  • Security effectiveness has improved on average since 2009 to 62% (default). With some default policies as low as 31%, tuning remains crucial for most solutions. Several vendors still failed the anti-evasion testing, leaving gaping holes in defenses.
  • Performance has decreased in general over the last year, with one vendor achieving just 3% of its claimed throughput.
  • For the first time, a few multifunction gateways are proving a credible alternative to stand-alone IPS products for mid-market deployments.

In the year since NSS Labs’ last IPS test, attackers have refined their strategy and have increased both the volume and the intelligence of their attacks. “Drive-by” downloads and exploits have been combined with disciplined attacks such as Operation Aurora, and the Zeus and Skynet botnets which target financial institutions. These test results point towards the need for organizations to continually evaluate their IPS options to make sure they are not overpaying for an underperforming solution.

NSS Labs compared the products head-to-head against 1,179 live, enterprise-class exploits using its real-world testing methodology. Products were tested using the vendor’s default or “recommended” settings and then again as tuned by a vendor representative. New in this year’s report is the Security Value Matrix (SVM), which allows enterprises to compare the cost and effectiveness of tested products on an apples-to-apples basis.

“Cyber criminals have all the time in the world to plan and attempt attacks. Our data and analysis are based on multiple man-years of complex, real-world testing that mimic how cyber-criminals are working to penetrate corporate defenses,” said Rick Moy, president, NSS Labs. “This report answers the critical questions on product capabilities and limitations that enterprises cannot answer without great effort and investment in time, equipment, and specialized expertise.”

All leading IPS vendors were invited to participate in the test at no cost. All testing was conducted independently and was not paid for by any vendor. Products tested in the report include:

  • Check Point Power-1 11065
  • Cisco IPS 4260
  • Endace Core-100 (IDS)
  • Fortinet Fortigate 3810
  • IBM GX6116
  • Juniper IDP 8200
  • Juniper SRX 3600
  • McAfee M-8000
  • Palo Alto Networks PA-4020
  • Sourcefire 3D 4500
  • Stonesoft IPS 1205
  • Stonesoft IPS 3205

The IPS Comparative Group Test Report is available now to NSS Labs’ subscribers. In addition, details of a specific product’s results are available in Individual Product Test Reports. All available IPS reports can be found here.

About NSS Labs, Inc.

NSS Labs, Inc. is the leading independent, information security research and testing organization. Its expert analyses provide information technology professionals with the unbiased data they need to select and maintain complex security products for their organizations. Pioneering intrusion detection and prevention system testing with the publication of the first such test criteria in 1999, NSS Labs evaluates firewall, unified threat management, anti-malware, encryption, web application firewall, and other technologies on a regular basis. The firm’s real-world test methodology is the only one to assess security products against live Internet threats. NSS Labs tests are considered the most aggressive in the industry. Founded in 1991, the company has offices in Carlsbad, California and Austin, Texas. For more information, visit