Power, Responsibility, Compliance, and Segmentation

Author: Andrew Lowe

“With great power there must also come great responsibility.” – Stan Lee, 1962.

It may not be what Stan Lee was talking about, but this philosophical phrase fits when talking about compliance. It is the compliance department’s responsibility to guide an organization in its pursuit to follow the regulations and/or guidelines set in place by law or by the organization itself. Compliance departments have the ability to enforce change and are sometimes thought to have the power to disrupt workflow. Despite what some may perceive, compliance teams are looking to improve individual department workflows, which in turn helps optimize their organizations.

One of the compliance team’s greatest responsibilities is to define the parameters of the program to be evaluated and audited, which is known as a scope. Compliance brings an advantage to organizations bidding for contracts and enhances marketing power and public relations building. However, compliance is costly, and organizations may not have the budget to evaluate an entire network due to the amount of audit data that has to be analyzed. I am often asked how an organization can scale down its audit scope while supporting an increasing number of compliance frameworks—and without adding analyst headcount.

One approach to solving the challenge of scaling is through network segmentation, a network access control technique that utilizes network technology to isolate a portion of a network for all the data that falls within compliance parameters. PCI-DSS is an example of where segmentation is useful—even required—as it is a portion of most businesses. Payment transactions have no place on the employee work area networks and should be isolated through segmentation.

Common technologies used to implement network segmentation include firewalls, SD-WANs, Layer 2 or 3 switches, and routers. In addition to the actual segmentation itself, logs from these devices are invaluable for auditors during evidence collection. They’re also helpful for security teams’ hardening efforts and incident response and for legal teams providing data to insurance companies if a breach does occur.

It’s a lot of power and a lot of responsibility. Keep the philosophical words of the late Stan Lee in mind and remember that as compliance and/or security specialists, we have great power but also great responsibility for our organizations’ and clients’ data. Network segmentation is important and very useful, but it is only one piece of the puzzle.

Read more about compliance and auditing implementations in my Intelligence Brief. For those of you in the midst of audit and compliance efforts, “Excelsior!”.

The NSS Labs 2019 Enterprise Intelligence Brief on Compliance and Auditing offers visibility into current enterprise requirements for regulatory and guideline frameworks, assessment phases, and clarifying terminologies. The paper is available to subscribers to our Research Library.

SD-WAN 2.0 Errata

In NSS Labs’ SD-WAN 2.0 testing, our graphical model represents the devices that were tested in our lab environment. Every vendor in the Network Value Map is represented based on both performance and costs of the solution submitted to satisfy the requirements in the methodology. In this test, VMware submitted its Edge 2000 devices, which were both tested and modeled with 1Gb licensing in the NVM; however, VMware believes that the 540 series is a better fit for the use case as depicted. NSS is committed to providing enterprises with actionable, accurate data and we will be looking forward to validating the 540 series capabilities and publishing a follow-on report supporting this recommendation by VMware.

In June 2019, NSS published the results of our SD-WAN v2.0 Group Test. This test included 8 vendors and was the most comprehensive SD-WAN test to date. Test reports are available to subscribers to our Research Library.


By Jason Pappalexis

Mobile devices remain one of the fastest evolving technologies within an enterprise’s IT security architecture. Advances in chip architectures, improved battery life, lower manufacturing costs, a consumer desire for “smart” products, quality on-board sensors, and enhanced APIs into corporate productivity suites have coupled with agile software development techniques and advanced manufacturing to enable new products to reach enterprise consumers at an unprecedented speed.

NSS Labs recently completed the 2019 NSS Labs Mobile Security Study in an effort to understand the use of these products in the enterprise. Data was obtained using a two-armed qualitative and quantitative study (n=383) with reach into both United States and European organizations. This data will be used to inform our enterprise client inquiry, develop test methodologies, and improve our overall understanding of IT security architecture risk.

We learned many things from this study, including the challenges associated with building a mobile security strategy; enterprise perceptions of risk, privacy, protection, and maturity; observed mobile threats; and the types of mobile devices capable of accessing corporate data. Some excerpts:

·       49.4% of respondents reported poor user awareness as the greatest challenge to mobile security strategy.

·       On average, survey respondents with mobile security rated their protection as 76.1 out of 100, while respondents without mobile security rated their protection as 70.1 out of 100.

·       More than half of all respondents reported that mobile threats were a higher risk to organizational assets than other cyber threats.

Any mobile device that can access corporate resources has the potential to introduce risk. The term “mobile device” has often been used to describe smartphones but today comprises a much longer list of products, including laptops, wearables, tablets, smartphones, and many other devices that not long ago would have been considered simply “IoT” by many. Recognizing these products as mobile devices is necessary if we are to understand the risk they introduce. This then allows IT security teams to choose the appropriate security products and to manage them effectively. While many executives acknowledge the risk associated with mobile devices, they also recognize the challenges of managing them. When asked for the top reasons they weren’t deploying a mobile security technology, the largest number of study respondents chose “mobile security is not a pressing need” and “privacy.”

Mobile security products are rapidly evolving to keep up with the pace of change. Mobile device management (MDM) was the earliest entrant to the space and targets traditional mobile operating system with features such as device locate, lock, and wipe. Mobile threat detection (MTD) focuses on vulnerability assessment, network security, application scanning, and URL filtering. Enterprise mobility management (EMM) provides the next iteration of MDM features, layering secure access to mobile applications across broader operating systems, and it is considered a transitional technology—a precursor to unified endpoint management (UEM). And finally, UEM, which is considered the current iteration of mobile security products with technology that supports broad anti-threat, identity, and device management features. UEM is interesting to IT security teams, as its operating system support has overlap with operating systems supported by endpoint security products (e.g., advanced endpoint protection products, or AEP ), forcing a broader discussion because UEM is no longer “smartphone only”.

What is an organization to do? Enterprises should not reduce their expectations for protection based on the challenges of mobile security technology. At some point in the future, enterprises will demand that an endpoint security product is capable of providing visibility into all endpoints capable of accessing corporate data, not just those marked as traditional operating systems.

NSS Labs has published a series of Intelligence Briefs on security controls in the US enterprise. The NSS Labs 2019 Enterprise Intelligence Brief on Mobile Security offers visibility into current enterprise requirements for the technology. The paper will be available to subscribers to our research library.

Read Our Responses to Your AEP 3.0 Questions

In March of this year, we published the reports from our Advanced Endpoint Protection (AEP) v3.0 Group Test. On April 26th, we presented our AEP 3.0 webinar, during which we looked at the test results, discussed our methodology approach, and provided AEP product selection guidance.

Our group test webinars aim to provide insight into important NSS testing concepts and help those unfamiliar with our testing process understand both what motivated a test and how the information can be used effectively. As such, they are mostly one-way communication until the Q&A at the end when there is opportunity for us to engage with our audience in a dialog of sorts. During the AEP 3.0 webinar, we received a number of great questions, some of which we were not able to address at the time. We thought it was important to provide a comprehensive follow-up—and let you know where to find more information. To this end, we have documented the questions and published our responses on an AEP Webinar Q&A page.

Here’s what our presenters had to say:

Scott Robin:

I was impressed with attendance numbers, both for the live webinar and for the recording. Many industries and sectors were represented, including government, construction, education, energy, financial, healthcare, and transportation. Audience members held a variety of roles: one third indicated they were managers and team leads, and one third indicated they held director, VP, executive, and C-level positions. Company size varied from less than 10 to 5000+. We received a wide range of questions from our diverse audience and we are following up across multiple areas including vendor/product selection advice, product tuning before and during testing, threats used during testing, test result distribution and timing, and the NSS Security Value Map (SVM).

Jason Pappalexis:

During the webinar, I mentioned that we are often asked which endpoint product is “the best.” My reply? “No such animal,” This is because every enterprise consumer of security technology has its own requirements. Common elements do exist, but it is up to an organization to first identify its priorities and then determine which products to explore further.

Use case, risk tolerance, and IT security architecture play an important part in any product selection decision. Use case because the systems the endpoint product will protect must be taken into consideration; risk tolerance because it is important to know what the impact will be if the system is not protected; and IT security architecture because the environment(s) the product exists in and data sharing requirements by adjacent security products are growing and thus becoming more important to consider.

We didn’t get into “secondary” factors during the webinar (and these are really primary factors once security effectiveness is determined). Such “secondary” factors include central management decisions (Should an organization choose cloud-delivered or on-premises deployments?) and interoperability concerns (Does the AEP product provide the information needed for adjacent controls to leverage the insight gained from the endpoint security product?).

Find out more

Visit our AEP Webinar Q&A page where we address questions asked during the presentation.

In March 2019, NSS published the results of our AEP v3.0 Group Test. This test included 21 vendors and was the most comprehensive endpoint test to date.

NSS has published a series of Intelligence Briefs on security controls in the US enterprise. The NSS Labs 2019 Intelligence Brief on Advanced Endpoint Protection provides visibility into current enterprise requirements for the technology. The paper is available to subscribers to our Research Library).

Secure SD-WAN is a great idea, but how much risk can you tolerate?

Author: Jason Pappalexis

In a world of multi-page product specification sheets, SD-WAN technology can be described in just one sentence: Network technology providing WAN link resilience, bandwidth, and control using network edge appliances (existing firewalls and routers or dedicated controllers) as termination points. The devil is in the details, however—especially considering the complexity of network routes and policies within a modern enterprise IT security architecture. In spite of this complexity, the “basic” SD-WAN is by many accounts a relatively straightforward technology to implement with a clear ROI.

Secure SD-WAN is an entirely different story. Add full security stack capabilities to the SD-WAN feature mix (think NGFW + WAN circuit management + WAN optimization), and deployment complexity ramps to a whole new level. Complexity considerations aside, it isn’t surprising that NSS Labs enterprise clients are investigating replacing branch office NGFW appliances with secure SD-WAN technology. Purchase drivers include vendor consolidation (i.e., enterprises want to simplify multi-vendor firewall deployments) and capex reduction (if secure SD-WAN meets organizational security requirements, hardware can be removed at branch offices without increasing risk).

It is important to maintain perspective amidst the hype. If an enterprise is looking to answer the question of whether or not it should replace its NGFW with secure SD-WAN technology, the answer depends on its tolerance for risk. To be successful as an NGFW replacement, secure SD-WAN technology must meet both firewall and network routing requirements—all requirements, not just high-level anti-threat or basic routing capabilities. Here, the “secondary” features (which are really the primary features when it comes to production use) make the difference: interoperability, policy management, diagnostics, alert handling, logging, management console workflow, signature quality, firmware development speed, capacity planning, etc. We can leave QoS off the table because it is assumed that if an SD-WAN cannot meet basic throughput, latency, jitter, and packet loss requirements, then it isn’t an option

Clearly, an organization must consider many factors before making the switch to secure SD-WAN technology. Even the definition should be evaluated prior to initiating a proof of concept; “security” in SD-WAN varies per vendor; some market it as encryption, others describe it as service chaining, and still others define it as full stack security. Our assessment is that, at least for now, SD-WAN products offered by firewall vendors are the safest choice for organizations intolerant to risk.

While secure SD-WAN technology offers today’s enterprises an enormous opportunity for cost savings, consolidation, and resilience, enterprises must understand all of the factors associated with a successful deployment—the cost advantages of this technology cannot be considered justification for increasing an organization’s risk tolerance. Our new Intelligence Brief on SD-WAN takes a closer look at this technology.

NSS Labs has published a series of Intelligence Briefs on security controls in the US enterprise. The NSS Labs 2019 Intelligence Brief on SD-WAN offers visibility into current enterprise requirements for the technology. The paper is available to subscribers to our research library.

Does your NGFW or NGIPS provide resilient vulnerability protection?

By Ty Smith, Jason Pappalexis

NSS Labs’ test methodologies are constantly evolving to reflect the needs of enterprise consumers. Last year, NSS introduced a “resilience” category to our security testing for inline devices (e.g., next generation firewalls [NGFWs], next generation intrusion prevention systems [NGIPS]). NSS defines resilience as a product’s capability to continue providing protection for a vulnerability against a known exploit after various modifications have been made to the original exploit.

Resilience is an important capability since exploit modifications often require little technical skill to implement. For example, a script-based drive-by exploit delivered in HTML may include a great deal of content that can be easily modified, such as the names of variables and functions; the order in which they are declared; adding and removing whitespace; swapping payload; changing comments; and/or a combination of the techniques just described.

Enterprises rely on network security products to provide protection for unpatched and/or vulnerable applications in their environments, picking off exploitation attempts “on-the-wire.” The goal of NSS’ resilience testing is to evaluate the quality of a product’s vulnerability protection signatures in order to determine whether they can adequately detect or match the essential exploitation elements of a vulnerability (in other words, the “trigger”), or can be easily bypassed by a relatively novice adversary making simple changes to a readily available POC exploit.

Inline security products are largely dependent on pattern-matching signatures (e.g., “match if this string of bytes is seen within x bytes of this other string of bytes”) to identify malicious content after normalization of network streams. These products are unlikely to further process the content (e.g., decode, deobfuscate, or render the HTML and associated script) [1] prior to inspection due to the impact it would have on throughput.

I wanted to share some resiliency results from our most recent NGFW v8.0 and NGIPS v4.0 public tests. In these tests, we used a well-known public drive-by exploit for CVE-2014-6332 (a.k.a. “Windows OLE Automation Array Remote Code Execution Vulnerability”). We chose this exploit because of the variety of content available for modification and for its reliability in delivering payloads to our intended victim (Microsoft Windows 7 SP1 with Internet Explorer 11).


Our set of test cases started with basic content transformation cases where a single technique was applied. Techniques were then stacked together, increasing the likelihood that we would modify or break up signature strings used by the device under test. Techniques utilized for this particular exploit included:

  • Adding whitespace (spaces and linefeeds)

  • Renaming of VBScript procedures and variables

  • Varying chr()/chrw()/chrb() VBScript function usage for character conversion

  • VBScript obfuscation with online tool (character/command conversion utilizing Execute()/chr()/CLng() functions)

  • Payload swapping/obfuscation

  • Combining/reordering of VBScript functions

  • String splitting/combining

  • Numeric value/equation modifications

In order to confirm that our traffic was being properly classified and normalized prior to inspection, we also delivered each of our test cases in chunked and/or compressed streams, and/or over a different TCP port.

As a result, during resilience testing 10 out of 10 NGFW products (10 of 10 participating vendors) were bypassed, and six out of seven NGIPS products (five of six participating vendors) were bypassed. Representative subsets of the resilience cases tested, along with total product bypass counts for each test case delivered over TCP port 80, follow (‘ P ’ = pass/block; ‘ F ’ = fail/miss):


Figure 1 – NSS Labs’ NGFW 8.0 Resilience Test Results


Figure 2 – NSS Labs’ NGIPS v4.0 Resilience Test Results [2]

As you can see, individual techniques generally had little impact. But as techniques were combined, the contents of the original exploit were obfuscated enough that pattern-matching signatures began to fail, as illustrated by the increase in bypasses. The results also indicate that some vendors’ signatures were more resilient than others.

While possible, further processing beyond normalization and categorization of streams prior to inspection is historically a relatively expensive proposition for these inline security products. An NGFW or NGIPS could go so far as to send files (including HTML) to the cloud or another on-premises sandbox for execution/deobfuscation and/or more extensive analysis before rendering a verdict to block or allow, but the delay would likely be considered unacceptable to the average user, particularly for something like web browsing. More extensive analysis to detect and block more complex attacks is generally reserved for other layers of a “defense-in-depth” security posture, such as endpoint, proxy, and/or sandbox security technologies, where the content can be more thoroughly normalized and/or detonated prior to inspection.

For NGFW and NGIPS devices, short of additional processing, pattern-matching signatures that identify heavily or suspiciously obfuscated content—not the underlying malicious content itself, can provide some additional protection but are also prone to false positives. Signatures that match fingerprinted delivery techniques and/or payloads by various exploit kits and threat groups can also be utilized. But the most resilient vulnerability protection signatures require extensive research and understanding of a CVE in order to write intelligent signatures targeting the essence/trigger of the vulnerability (relatively expensive but resilient), as opposed to simply adding strings seen in public exploits to a list to pattern-match against (relatively cheap but not resilient).

So, what’s the takeaway? While our resilience testing reveals some limitations, NGFW and NGIPS technologies remain an important part of a “defense-in-depth” security strategy. Our testing indicates which vendors are making the requisite investment in time and expertise to write signatures that offer the most resistance to exploit modifications. Combined with performance and other evasions testing with the same device configurations, we can determine which products offer not only the most effective security, but also which are most efficient at doing so. For additional details, including which vendors missed which resilience cases, individual test reports are available at research.nsslabs.com.

[1] Additional NGFW JavaScript obfuscation testing data is available here: Wait, I thought my NGFW Detected That

[2] While the NGIPS device from Vendor 5 blocked all test cases delivered over TCP port 80, nearly every test case delivered over a specific non-standard TCP port bypassed the device.

The NGFW today: A staple of network security in spite of challenges

By Jason Pappalexis

Some technologies are revolutionary—so much so that in less than a generation, it is difficult to imagine living without them. Take, for instance, the automobile. In 1908, the first production Model T Ford was completed. By 1927, close to 15 million had been built, steadily approaching the “ . . . a car in every garage” goal set by Henry Ford himself (in the 1920s, there were likely far fewer garages). Fast forward to the smartphone—first released in the early 2000s, they are now ubiquitous—in the hands of everyone from toddlers to the elderly.

Today, cars and cellphones are pretty much considered essential for modern living. Can the same be said for current next generation firewall (NGFW) technology within IT security architectures?

As the logical evolution of the traditional packet inspection firewall, the NGFW grabbed hold of the prime spot at the network edge, and it hasn’t let go. And for good reason—its broad feature set provides value across organizational teams; for example, network operations (tunneling, routing, etc.), IT security (inspection, application control, decryption, etc.), and even human resources (per user behavior control through URL categorization, etc.). NGFWs are flexible products, which makes them “sticky” in terms of deployment.

However, the technology is not without challenges. When multi-function systems are deployed within latency-sensitive environments, accurate capacity planning is critical. Given the number of variables within enterprise traffic and general uncertainty about how activating NGFW features will impact performance, many organizations oversize. (Nearly one third of respondents in the 2018 NSS Labs Network Security Study (1) indicated that their organization targets 50% above peak throughput requirements for sizing.) This can be expensive. What’s more, enterprises are so latency averse that it is not uncommon for devices to be configured in monitor mode in order to further reduce their risk‑-but this is at the cost of security.

There is also uncertainty regarding a product’s actual security effectiveness. The largest proportion of respondents in the 2018 NSS Labs Network Security Study reported their minimum acceptable security efficacy is in the range of 95% to 99%,(2) but expectations of protection and actual protection do not appear to always align. NSS’ 2018 testing reveals inconsistent average exploit block rates over time (3) and mishandling of exploits delivered by web-based scripts.(4) Expect to see the number of script threats increase as their success rate becomes more widely known. (e.g., Trend Micro recorded a spike in malicious JavaScript in January 2019; the spike reached 55.4% in Japan and 14.7% in the US.)

Cloud form factors (both virtual appliance and as a service) introduce more unknowns. Do these form factors provide the same protection as traditional on-premises appliances? Perhaps more importantly, do they protect threat vectors that are uniquely associated with the enterprise use of cloud resources? While cloud NGFW technologies are being considered by enterprises for deployment in their IT security architectures, the market is still young and actual deployments vary. In the future, enterprise requirements for cloud-based and cloud-delivered NGFWs are likely to have a significant impact on the growth of this technology.

While the challenges discussed here are not inhibiting NGFW adoption, they do provide an opportunity for vendors to set realistic expectations and inform enterprise customers of product gaps, which will help the customers plan properly and reduce risk. Network inspection technology remains essential in today’s IT security architectures, and it is important for enterprises to understand the technology’s capabilities as it continues to evolve.

NSS Labs has published a series of Intelligence Briefs on security controls in the US enterprise. The NSS Labs Intelligence Brief on NGFW offers visibility into current enterprise requirements for the technology. The paper is available to subscribers to our research library.

  1. 2018 NSS Labs Network Security Study was conducted in the Fall of 2018 and targeted 151 full-time US enterprise IT security professionals representing 28 US industries with a median IT security budget of US$10M – $50M.

  2. The largest proportion of respondents (31.1%) in the 2018 NSS Labs Network Security Study reported minimum efficacy in range 95% to 99%.

  3. 2018 NSS Labs Evolution of Product Testing: Firewall

  4. 2018 NSS Labs Investigative Report: The Impact of Code Obfuscation and Web Delivery Encoding on NGFW Scanning Accuracy