Comparing cybersecurity products isn’t something to be taken lightly. There are a lot of factors to consider, and one of the biggest hurdles consumers will have to overcome is bias. Every company will spin things to persuade customers that its product is the best choice. Whether investing in something as inconsequential as a pair of sneakers or something as significant an enterprise software product, consumers can face tough choices. In just about every case, teams will spin content so that their products are viewed in the most favorable light. It’s just the way the marketing world works.
Enterprise security teams making decisions about cybersecurity are often left to make choices based on limited, biased information.
The best way to remove the bias is through testing. Enterprise security teams conduct trials, but resource constraints often result in them testing only two to three vendors out of 10-50+ possible choices. And these trials are limited in scope since testing isn’t a core competency. Additionally, the test traffic used typically isn’t representative of all of the different exploits, so results tend to swing toward the product the tester learned the fastest or the product the sales engineer spent the most time with.
Not exactly a science, is it?
Erasing bias means looking at test labs that specialize in the technology being tested. That’s why dedicated test labs exist—they have the resources to execute tests effectively. These labs help narrow product selection to the two to three products that you should conduct trials on, instead of the two to three with awesome marketing.
When checking out test labs, look for the following:
Expert Guidance – Testing cybersecurity is not the same as network testing. There are different elements to consider, such as encryption and exploits —and you have to understand if a product is proactive or reactive.
Transparency – Trust is a big factor when making a choice. You should have access to a detailed, easy-to-read methodology that’s based on best practices. If a test lab isn’t willing to share information, chances are something sketchy is at work.
Empirical Data – Having access to data can remove bias. And while all test results provide data, it is the “real” data that matters, such as information on a product’s security effectiveness.
Objective Analysis – Remaining vendor agnostic allows for unbiased, objective analysis. Private tests don’t introduce bias since results generally are seen only by the vendor whose products are being tested. But for public tests, vendors want results that reflect favorably on them, and this can introduce bias. To remove the potential for bias, public tests should be performed for free.
Understanding how bias works in testing will help you with your decision making. Choosing the right security for your company isn’t easy, but NSS Labs is here to help make sure you choose the best one for your organization.
