How well does “defense in depth” really work?

Defense in Depth“Defense in depth,” or the layering of multiple security products is a commonly employed security strategy and best practice. Central to the concept of layered security is the idea that attacks that are able to bypass one layer of security will eventually be caught by a subsequent layer of security.

In a first order approximation, the effectiveness of this approach is typically calculated as the product of the individual layer’s failure rate. For example, if layer 1 is assumed to miss 10% of the attacks and layer 2 is assumed to miss 10% of the attacks, then the combined failure rate of these two layers is estimated to be 10% x 10% = 1%.

So much for the theory, how effective is this approach in practice?

To answer this question, we looked at the results of group tests conducted by NSS Labs over the past 18 months, where NSS tested the security effectiveness of typical defense technologies that are deployed in layered security, such as next generation firewall (NGFW), intrusion prevention systems (IPS), and endpoint protection (EPP) – also referred to as antivirus/malware detection. We used real-world attack scenarios to measure the exploit-blocking abilities of 37 security products from 24 different security vendors. The 1,711 exploits that were used in these tests target 816 software products from 208 different software vendors, thereby covering 21 percent of all vulnerabilities published against these software products in the last 10 years.

The results of these group tests provide a unique and wonderful set of data to assess not only the detection performance of individual devices, but also the performance of any combination of security devices. Our analysis reveals a significant correlation of failures to detect exploits within and across multiple defense technologies.

Key findings of our research:

  • Security performance varies greatly not only between individual products, but also between combinations of specific products
    The joint failure rate for all combinations of security device pairs was lower than the failure rate of any single device; however, the exact combination of products makes a significant difference. The best combination of two IPS devices, for example, detected all but 2 exploits, while the worst combination failed to detect 61 exploits.
  • There is only limited breach prevention available
    We examined 606 unique combinations of security product pairs (IPS + NGFW, IPS + IPS, etc.) and only 19 combinations (3 percent) were able to successfully detect ALL exploits used in testing. This correlation of detection failures shows that attackers can easily bypass several layers of security with the use of only a small set of exploits.
  • Exploits bypassing detection attack prevalent and relevant software, not niche products
    The exploits that bypass the most systems almost exclusively target software from mainstream software vendors that is used regularly within the enterprise and private environments. For example, none of the 33 network security devices (NGFW and IPS) that were tested were able to successfully detect all exploits against Microsoft products, and only 5 of the 33 systems tested were able to successfully detect all exploits against Apple products.
  • Correlation matters
    The number of exploits that were found to bypass multiple security devices is significantly higher than the common expectation, or than the prediction of simple risk models that ignore the effects of correlation by simply multiplying individual failure rates. Security professionals must take into account the effects of correlation when modeling risk.

We were surprised to find that a mere 3% of the 606 unique security product combinations that were tested were able to detect all exploits. Ignoring this correlation leads to an overestimation of the security effect of combining multiple protection technologies, by orders of magnitude.

Layered security is beneficial when looking to secure the enterprise; however, it is the choice of security devices to be combined that is key to realizing substantial security gains and to offsetting the increase in complexity, management, and cost.

The full analysis is freely available in the brief “Correlation of Detection Failures”.

Follow me on Twitter @stefan_frei to stay informed about our latest groundbreaking research.

Go to top