7 of 9 Next Generation Firewalls Tested Received NSS Labs Coveted “Recommend” Status

AUSTIN, Texas – February 26, 2013 – NSS Labs today released its 2013 Next Generation Firewall (NGFW) Security Value Map and Comparative Analysis Reports, which evaluated 9 of the leading NGFW products on the market for security effectiveness, performance, enterprise management capabilities and total cost of ownership. This was the second group test for NGFW that NSS has conducted and overall there was marked improvement from most vendors’ 2012 test scores.

NSS’s research yielded several key conclusions:

• Check NGFWs’ firmware before deployment: Out of a total of 9 products tested, 6 vendors submitted products that required firmware updates or configuration changes to complete the NSS tests. Only Check Point, Fortinet and Stonesoft submitted products that worked the first time.
• New Metric Highlights Enterprise Management Failings: If a device cannot be managed effectively, the security effectiveness of that device is compromised. As part of this test, NSS performed in-depth technical evaluations of all the main features and capabilities of the enterprise management systems offered by each vendor and factored it into the final score as a new and unique metric called “managed security effectiveness”. Managed security effectiveness scores ranged from 29.1% to 98.5%.
• NGFWs’ Security Effectiveness Scores Improve Significantly: In the latest 2013 tests, 8 of the 9 products scored over 90% for security effectiveness (excluding management). This is a marked increase compared to 2012, when only half of tested vendors scored above 90% in this category. The overall scores for security effectiveness in 2013 ranged from 34.2% to 98.5% compared to 18% to 98.9% in 2012.
• Total Cost of Ownership Remains Fairly Stable: While the overall range of TCO decreased in 2013 testing, prices per protected megabit per second remained fairly stable with most tested devices costing below $44 per Protected-Mbps. The overall 2013 range was $18 – $124 per Protected Mbps, down from a range of $30 – $375 in 2012 testing.
• More Vendors Back their Performance Claims: Only 2 of 9 products tested had throughput rates that were significantly less than their vendors’ stated claims. In 2012 testing, 5 of the 8 products tested performed well below their advertised speeds. In 2013, three vendors – Dell SonicWALL, Sourcefire and Palo Alto – performed better in tested performance than their stated throughput and two vendors – Check Point and Stonesoft – had throughputs that were virtually equal to their stated performance.

Commentary: NSS Labs Research Director Francisco Artes

“In 2012, our tests showed that while vendors turned in a good first showing, there was significant room for NGFW technologies as a whole to improve before being widely deployed in large enterprises,” said Francisco Artes, Research Director at NSS Labs. “In our 2013 tests, I think we’ve seen much of the improvement we thought was needed in previous testing. With 7 of the 9 products receiving a `Recommend’ rating in this year’s tests, it’s clear that the vendors are investing a lot of time and effort to address many of the overall stability, leakage, performance and security effectiveness concerns from last year.”

The 2013 NGFW Security Value Map™, Comparative Analysis Reports™, and Product Analysis Reports™ for each vendor are currently available to NSS Labs’ subscribers at nsslabstage.wpengine.com.

The products covered in the 2013 NGFW Group Test are:

• Check Point 12600
• Dell SonicWALL SuperMassive E10800
• Fortinet FortiGate 3600C
• Juniper SRX 3600
• Palo Alto PA-5020
• Sourcefire 8250
• Sourcefire 8290
• Stonesoft 3202
• WatchGuard XTM 2050