NSS Labs Announces 2019 NGFW Group Test Results

Next generation firewalls are core to many cybersecurity strategies, and yet most of these products were easily evaded in this test.

AUSTIN, Texas – July 17, 2019 – NSS Labs, Inc., a global leader and trusted source for independent cybersecurity product testing, today announced the results of its 2019 Next Generation Firewall (NGFW) Group Test. Twelve of the industry’s leading NGFW products were tested to compare NGFW product capabilities across multiple use cases. Products were assessed for security effectiveness, total cost of ownership (TCO), and performance.

Firewalls are the most widely deployed network security devices. Enterprises expect modern firewalls (NGFWs) to prevent exploits and malware from infecting critical systems.

This is the ninth year for testing NGFW products. NSS raised the bar this year by performing a significantly harder test for security effectiveness, which exposed weaknesses not seen previously. Test results showed that block rates for simple clear-text attacks remain strong (over 96%) for nine out of twelve products. However, while known/published exploits were frequently blocked, test engineers were able to bypass protection in all devices with minor modifications to known and blocked exploits. In addition, only one of twelve products properly blocked exploits that were obfuscated using Complex Evasions (HTML / JavaScript / VBScript). Palo Alto Networks and WatchGuard stood out as the only products that didn’t miss major evasions this year.

Key Takeaways:

·       Enterprises expect when they purchase products that they will remain viable over multiple years.

·       While it is tempting to draw conclusions from one test, NSS recommends enterprises favor vendors that consistently engage and improve over time. When in doubt, an NSS analyst is available to answer questions.

·       Scripting evasions are challenging for NGFWs since they require real-time code analysis in order to determine whether a function is legitimate or obfuscating an attack.

·       Vendor claims to protect vulnerabilities (regardless of the exploit specifics) are largely dependent on the nature of the vulnerability and whether it lends itself to such protection. Test results found all products had room for improvement when confronted with unknown variants of known exploits.

·       Research indicates that over 70% of Internet traffic is encrypted using TLS/SSL. NSS recommends measuring the performance of devices both with and without TLS/SSL enabled. Failure to do so could result in unexpected performance bottlenecks.

“Given the ever increasing integration of the cyber and physical world, it is imperative that cybersecurity products work properly,” said Jason Brvenik, Chief Executive Officer at NSS Labs. “The good news is that while we found flaws, most vendors are committed to protecting their customers and are fixing their products. Stay tuned for follow-on reports,” added Brvenik.

Of the twelve products tested, ten were rated as Recommended based on comparative scores for overall security effectiveness, TCO per protected Mbps, and performance:

·       Barracuda Networks CloudGen Firewall F800.CCE v7.2.3

·       Check Point Software Technologies 6500 Security Gateway R80.20

·       Forcepoint 2105 NGFW v6.3.11

·       Fortinet Fortigate 500E v6.0.4 build 0231

·       Huawei USG6620E v600R006C00SPC310

·       Palo Alto Networks PA-5220 PAN-OS 8.1.6-h2

·       Sophos XG 750 Firewall SFOS v17.5

·       SonicWall NSa 4650 SonicOS v6.5

·       Versa Networks FlexVNF v16.1R2-S7

·       WatchGuard Firebox M670 Firmware: 12.3 B589695 Ver-4.907

NSS Labs is committed to providing empirical data and objective group test results that help organizations make educated decisions about purchasing and optimizing security products and services. We believe if a product is good enough to sell, it is good enough to test. If you do not see a product you are interested in, ask them where their results are and encourage participation. As with all NSS Labs group tests, there is no fee for participation.

 

Additional Resources

·       View the 2019 NGFW Test Security Value Map (free)

·       View the 2019 NGFW Group Test Methodology (free)

·       Subscribers can access the 2019 NGFW Group Test reports here

·       View the Intelligence Brief on Security Controls in the US Enterprise: Next Generation Firewall (March 2019)

·       Learn more about how NSS Labs approaches testing

·       To suggest a product for testing, click here

·       Follow NSS Labs on Twitter

·       Follow NSS Labs on LinkedIn

###

 

About NSS Labs, Inc.
NSS Labs tests the world’s security products. Based in Austin, Texas, the company’s research and testing laboratory is recognized globally as the most trusted source for independent, fact-based cybersecurity guidance. C-Suite executives and information security professionals from many of the world's most demanding global enterprises rely on NSS Labs to accelerate security decisions with greater confidence. For more information, visit www.nsslabs.com.

 

Contact:

Jessica Johannes

Phone: +1 512-498-7076

jjohannes@nsslabs.com

Mobile Security – Small Devices, Big Challenges

By Jason Pappalexis

Mobile devices remain one of the fastest-evolving technologies within an enterprise’s IT security architecture. Advances in chip architectures, improved battery life, lower manufacturing costs, a consumer desire for “smart” products, quality on-board sensors, and enhanced APIs into corporate productivity suites have coupled with agile software development techniques and advanced manufacturing to enable new products to reach enterprise consumers at an unprecedented speed.

NSS Labs recently completed the 2019 NSS Labs Mobile Security Study in an effort to understand the use of these products in the enterprise. Data was obtained using a two-armed qualitative and quantitative study (n=383) with reach into both United States and European organizations. This data will be used to inform our enterprise client inquiry, to develop test methodologies, and to improve our overall understanding of IT security architecture risk.

We learned many things from this study, including the challenges associated with building a mobile security strategy; enterprise perceptions of risk, privacy, protection, and maturity; observed mobile threats; and the types of mobile devices capable of accessing corporate data. Some excerpts:

·       49.4% of respondents reported poor user awareness as the greatest challenge to mobile security strategy.

·       On average, survey respondents with mobile security rated their protection as 76.1 out of 100, while respondents without mobile security rated their protection as 70.1 out of 100.

·       More than half of all respondents reported that mobile threats were a higher risk to organizational assets than other cyber threats.

Any mobile device that can access corporate resources has the potential to introduce risk. The term “mobile device” has often been used to describe smartphones but today comprises a much longer list of products, including laptops, wearables, tablets, smartphones, and many other devices that not long ago many would have considered simply “IoT”. Recognizing these products as mobile devices is necessary to understand the risk they introduce, which then allows IT security teams to choose the appropriate security products and to manage them effectively. While many executives acknowledge the risk associated with mobile devices, they also recognize the challenges of managing them. When asked for the top reasons they weren’t deploying a mobile security technology, the largest number of study respondents chose “mobile security is not a pressing need” and “privacy”.

Mobile security products are rapidly evolving to keep up with the pace of change. Mobile device management (MDM) was the earliest entrant to the space and targets traditional mobile operating system with features such as device locate, lock, and wipe. Mobile threat detection (MTD) focuses on vulnerability assessment, network security, application scanning, and URL filtering. Enterprise mobility management (EMM) provides the next iteration of MDM features, layering secure access to mobile applications across broader operating systems, and it is considered a transitional technology—a precursor to unified endpoint management (UEM). And finally, UEM, which is considered the current iteration of mobile security products with technology that supports broad anti-threat, identity, and device management features. UEM is interesting to IT security teams, as its operating system support has overlap with operating systems supported by endpoint security products (e.g., advanced endpoint protection products, or AEP ), forcing a broader discussion because UEM is no longer “smartphone-only”.

What is an organization to do? Enterprises should not reduce their expectations for protection based on the challenges of mobile security technology. At some point in the future, enterprises will demand that an endpoint security product is capable of providing visibility into all endpoints capable of accessing corporate data, not just those marked as traditional operating systems.

NSS Labs has published a series of Intelligence Briefs on security controls in the US enterprise. The NSS Labs 2019 Enterprise Intelligence Brief on Mobile Security offers visibility into current enterprise requirements for the technology. The paper will be available to subscribers to our research library.

NSS Labs Announces 2019 SD-WAN Group Test Results

All products tested met the use case requirements and offer a good ROI

AUSTIN, Texas – June 19, 2019 – NSS Labs, Inc., a global leader and trusted source for independent cybersecurity product testing, today announced the results of its 2019 Software Defined Wide Area Network (SD-WAN) Group Test. In this year’s test, eight of the industry’s leading SD-WAN products were examined to help enterprises understand the merits of products in the market and identify the capabilities best suited to meet their use case requirements. Products in this test were assessed for the quality of experience (QoE) of VoIP and video, performance, total cost of ownership (TCO) and security effectiveness.

NSS Labs defines SD-WAN as the union of software-defined networking (SDN) and WAN technology. Part router, part WAN optimization, and part firewall, SD-WAN enables enterprises to leverage high-bandwidth, consumer-grade links (or links without guaranteed performance) for business-class services at a lower cost than traditional dedicated links. Enterprises are adopting SD-WANs for their branch office network needs – capitalizing on the visibility, scalability, performance, and control benefits the technology provides. 

SD-WAN technology is rapidly evolving, and enterprises exploring SD-WAN technology should focus on network functionality and interoperability maturity to differentiate products. Additionally, SD-WAN is consumable in multiple ways including both appliance or as subscription based services. This flexibility offers enterprises the choice to deploy and manage their SD-WAN as business needs and IT priorities dictate. 

For this second iteration of the SD-WAN Group Test, NSS Labs identified three use cases: manageability and cost, performance, and security in the form of protection against network-delivered exploitation.

Key Takeaways

  • Products in this years test performed well against a rigorous series of test cases that capture efficacy in demanding WAN deployments. All products met the use case for quality of experience (QoE) for VoIP and Video, scoring above the minimum recommended by NSS Labs.

  • Traditionally, the SD-WAN market has been dominated by pure play SD-WAN vendors. In the last few years, security vendors offering threat protections entered the space, now offering a combined firewall/IPS protection with SD-WAN.

    • Security vendors already offer protection against network-delivered exploitation.

    • Two products with built-in protection against network-delivered exploitation capabilities were tested with the functionality enabled; both products performed equal to or better than products without this functionality enabled. Products with the capabilities but choosing not to enable them for testing should be fully assessed before purchase and deployment.

  • Two differentiators in this test were TCO per Mbps and protection against network-delivered exploitation.

  • On average, vendors who were tested with network-delivered exploitation protection as part of their SD-WAN had a lower TCO per Mbps than those who did not.

  • All tested products had performance-to-cost ratios that were better than Multiprotocol Label Switching (MPLS) or dedicated links, making a strong case for deployment of these SD-WAN products.

    • Most tested vendors had a simplified branch office configuration creation capability and feature measured deployment time of less than 10 minutes per device, demonstrating a positive impact on business expansion and productivity over legacy network solutions.

“Interest and demand for SD-WAN continues to accelerate as enterprises gain significant cost and operational benefits,” said Jason Brvenik, Chief Executive Officer at NSS Labs. “All of the products tested in the 2019 SD-WAN Group Test offer a notable return on investment. Products offering integrated threat protections demonstrate further return on investment and operational efficiency. We encourage enterprises exploring SD-WAN technologies to read the results from this year’s test.”

Products Tested:

·       Barracuda Networks Barracuda CloudGen Firewall F82 v7.2.3

·       Citrix Systems Citrix SD-WAN 2100-1000-SE v10.0.0.6

·       Forcepoint NGFW 1101 SMC 6.5.3 and Engine 6.5.2

·       Fortinet FortiGate 61E v6.0.4 GA Build 0231

·       Oracle Talari SD-WAN E1000 v7.3

·       Silver Peak Unity EdgeConnect EC-M VXOA 8.1.7

·       Versa Networks FlexVNF V220 v16.1R2-S6

·       VMware SD-WAN by VeloCloud Edge 2000 v3.2.1

NSS Labs is committed to providing empirical data and objective group test results that help organizations make educated decisions about purchasing and optimizing security products and services. We believe if a product is good enough to sell, it is good enough to test. If you do not see a product you are interested in, ask them where their results are and encourage participation. As with all NSS Labs group tests, there is no fee for participation.

Additional Resources

·       View the 2019 SD-WAN Group Test Network Value Map (free)

·       View the 2019 SD-WAN Group Test Methodology (free)

·       Subscribers can access the 2019 SD-WAN Group Test reports here

·       View the Intelligence Brief on Security Controls in the US Enterprise: Software Defined Wide Area Network

·       Learn more about how NSS Labs approaches testing

·       To suggest a product for testing, click here

·       Follow NSS Labs on Twitter

·       Follow NSS Labs on LinkedIn

###

About NSS Labs, Inc.
NSS Labs tests the world’s security products.  Based in Austin, Texas, the company’s research and testing laboratory is recognized globally as the most trusted source for independent, fact-based cybersecurity guidance.  C-Suite executives and information security professionals from many of the world's most demanding global enterprises rely on NSS Labs to accelerate security decisions with greater confidence. For more information, visit www.nsslabs.com.

 

Contact:

Jessica Johannes

Phone: +1 512-498-7076

jjohannes@nsslabs.com  

NSS Labs Appoints New Chief Executive Officer

AUSTIN, Texas – May 2, 2019 – NSS Labs, Inc., a global leader and trusted source for independent cybersecurity product testing, today announced that Jason Brvenik has been named Chief Executive Officer (CEO); he had been serving as Chief Technology Officer (CTO) since January 2017.  

Vikram Phatak, CEO since 2007, will remain active on the Board of Directors and Executive team as Founder. Phatak has a deep knowledge of the cybersecurity industry and will focus on new innovations for the company. 

Under the leadership of Phatak, NSS Labs established itself as the independent trusted third party that understands both the needs of the enterprise and the true capabilities of the world’s cybersecurity products. Phatak recruited Brvenik to grow the testing programs and deliver customized proof-of-concept testing and product selection for enterprises. Over the past two years, Phatak and Brvenik have broadened services to help security professionals navigate the complexity and hidden disparity in security product selection and deployment.   

“Working with Jason has been very rewarding. He has recruited top talent and guided that team to significantly expand our security testing programs and the value we add to enterprises,” Phatak said. “He is definitely the right person for the job.”  

Prior to NSS Labs, Brvenik worked in technology and leadership roles at Sourcefire from 2002 until the company’s $2.7B acquisition by Cisco in 2013, where he also served on the security leadership team.

NSS Labs’ rigorous group tests offer independent analysis of the top security technologies used by Global 2000 companies and governments around the world. Consumers rely on NSS Labs’ fact-based, empirical data to inform their decision making. The company most recently introduced coverage of cloud security technologies and a Threat Detection and Analysis Systems group test.

“It has been an honor to work alongside Vik to help customers identify the technologies that are most effective in defending against the threats they face,” Brvenik said. “I am excited to continue our journey and help make truly effective security a reality for consumers.”

About NSS Labs, Inc.

NSS Labs tests the world’s security products.  Based in Austin, Texas, the company’s research and testing laboratory is recognized globally as the most trusted source for independent, fact-based cybersecurity guidance.  C-Suite executives and information security professionals from many of the world's most demanding global enterprises rely on NSS Labs to accelerate security decisions with greater confidence. For more information, visit www.nsslabs.com.

Contact:

Jessica Johannes

Phone: +1 512-498-7076

jjohannes@nsslabs.com  

NSS Labs to Develop the 2019 Threat Detection and Analytics Systems Group Test

TDA Represents an Evolution of the Original Breach Detection Systems Group Test

AUSTIN, Texas – April 3, 2019 – NSS Labs, Inc., a global leader and trusted source for independent, fact-based cybersecurity guidance, today announced that it is developing its Threat Detection and Analytics Systems (TDA) Group Test with results to be released in 2019. As part of today’s announcement, the company is also issuing a call for industry engagement from both enterprises and vendors that offer threat visibility and automation and response capabilities through the use of analytics to help shape and evolve the upcoming group test and accompanying methodology. 

While enterprises aspire to attain a perfect security architecture, the reality is that weaknesses can stem from a number of factors. These can include configuration error, lapses in operational hygiene, user error, threat and evasion capabilities, and malicious insiders. Increasingly, enterprises are turning to threat detection analytics technology to address evolving use cases for analytic capabilities to identify, investigate, and respond to incidents before a major incident or breach occurs.

Threat detection and analytics products improve the incident responders’ ability to rapidly assess and identify threat activities that incorporate subtle and advanced attack techniques that can bypass individual security controls unless examined across the attack sequence. Through the application of analysis algorithms and both traffic and often endpoint technologies, TDA technologies help to accelerate the response workflow and improve incident outcomes by correlating data across many data surfaces. Incident responders are uniquely able to address attacks in progress and help organizations avoid serious data loss or damage if they learn of incidents early enough in the attack chain and have sufficient detail to prioritize and act on threats.

This forthcoming test will evaluate both traditional TDA products and new entrants striving to address evolving enterprise use case requirements for this technology. Some of the capabilities this test will examine include enhanced identification of false positive events, detection of malicious activity or content, and operational and workflow impacts such as a product’s ability to streamline enterprise operations by integrating with other security tools.

In 2018, NSS Labs performed the industry’s most comprehensive group test of leading breach detection system products. Three products from market-leading vendors were examined for security effectiveness, performance, and total cost of ownership. Of the products that participated in the group test, only one product demonstrated full resilience tested against attack variants. For more information about the test, click here.

“The TDA group test will help enterprises evaluate whether to replace or refresh existing BDS deployments with TDA products or investigate new approaches that incorporate analytics and advanced feature sets,” said Jason Brvenik, Chief Technology Officer at NSS Labs. “We encourage both enterprises and vendors to collaborate with us as we examine this evolving category.” 

NSS Labs has a long history in testing enterprise-class security products. NSS Labs’ rigorous group tests offer independent analysis of the top security technologies used today by Global 2000 companies. The tests provide the industry’s most comprehensive review of security effectiveness, performance, and total cost of ownership. Enterprises rely on our tests for fact-based, empirical data that they can use to inform their decision making. Within the last 12 months, NSS Labs has released group test results for several categories of mature and evolving cybersecurity products. To learn more about our group tests, visit the NSS Labs website.

As with all NSS Labs group tests, there is no fee for participation, and the test methodology is available in the public domain to provide transparency and to help enterprises understand the factors behind test results. Click here for more information about our group test policies.

Enterprises that wish to provide feedback regarding NSS Labs’ upcoming TDA Group Test and its associated test methodology can send feedback to enterprise_relations@nsslabs.com. Vendors can send feedback to vendor_relations@nsslabs.com.

 

Additional Resources: 

·       Visit the NSS Labs website

·       Follow NSS Labs on Twitter

·       Follow NSS Labs on LinkedIn

### 

About NSS Labs, Inc.
We test the world’s security products. Based in Austin, Texas, our research and testing laboratory is recognized globally as the most trusted source for independent, fact-based cybersecurity guidance. C-Suite executives and information security professionals from many of the world's most demanding enterprises rely on NSS Labs to accelerate security decisions with greater confidence. For more information, visit www.nsslabs.com.

 

Contact:

Jessica Johannes

Phone: +1 512-498-7076

jjohannes@nsslabs.com