Competitive Benchmarks: AV-Test.org

Competitive Benchmarks - AV-Test

Who is AV-Test.org?

The company AV-Test GmbH is a worldwide operating and leading service provider for IT security testing and consultancy services. Their team has more than 15 years of experience in the area of anti-virus research and data security. Every year they perform more than 2,500 product tests of anti-virus, anti-spyware, personal firewalls and related products on behalf of vendors, integrators (OEM), corporate users and magazines.

Enterprise Endpoint Testing Results

May 2011

In May 2011, AV-Test.org again performed endpoint security benchmark testing on five market-leading Enterprise endpoint solutions from Symantec, McAfee, Microsoft, Sophos, and Trend Micro. For the twelfth straight month, OfficeScan has outperformed its peer competitive products in the overall results, Exposure Layer results and time to protect results. For a historical view of these test results, check out the tab labeled “Historical Results”.

Testing Methodology

Traditional tests have focused on identifying the product that can best detect a set of known malware files during a closed test. This type of testing is outdated and does not take into account how threats typically propagate today, or how computers are used. In this test, AV-Test.org utilized a testing methodology that is more real-world focused in which the samples are malicious URLs with associated malware files and allowed the solutions to block threats at their source (Exposure Layer), during download (Infection Layer) or on execution (Dynamic Layer). Also tested is the ability to source, analyze and protect against any samples that went undetected during the first round by re-testing 1 hour later.

Note: Cloud-based protection components were enabled for all products if not activated by default. This includes features like Trend Micro's web reputation service.

Trend Micro outperforms all other vendors

Download report >

Total percentage of threats blocked by all layers:
Exposure, Infection, Dynamic

Overall Blocking Rate

Source: AV-Test.org | Note: Results are based on the T+60 minute results. Test results can vary over time and may vary with other configuration settings.

And in the time to protect category, Trend Micro ranked #1 too.

Time to protect improvement percentage
% of previously unkown threats blocked at T = 60 minutes

Time to Protect

Source: AV-Test.org | NOTE: Time-to-protect improvement is the percentage of threats missed at T=0min that are subsequently prevented at T=60min. For example, with Trend Micro OfficeScan: At T=0min, 187 threats were prevented while 13 threats were missed. Of the 13 threat missed at T=0min, 3 were prevented at T=60min (3 of 13 equals 23.1%).

Why Trend Micro outperformed the competition

Trend Micro has outperformed the competition in a number of recent tests (See NSS Labs and Dennis Technology Labs) due to the Smart Protection Network™ infrastructure that powers products for consumer, SMB, Enterprise, Partner and SaaS customers but also powers our mobile, endpoint, server, messaging, gateway and SaaS solutions. This unique cloud-client architecture that uses Smart feedback to source new potential threats, patent-pending correlation to analyze multiple threat vectors (web, email, file) in real-time and in-the-cloud reputation databases to deliver protection wherever our customers connect, allows Trend Micro to react faster than other vendors against new threats. Our cloud-based protection network is the most evolved and comprehensive network in the industry blocking billions of email, web and file-based threats targeting our customers daily.

 

SMB Endpoint Testing Results

May 2011

In May 2011, AV-Test.org performed endpoint security benchmark testing on five market-leading SMB endpoint solutions from Symantec, McAfee, Sophos, ESET, Kaspersky, and Trend Micro. The Trend Micro Worry-Free Business Security outperformed its peer competitive products in the overall results and Exposure Layer results.

Testing Methodology

Traditional tests have focused on identifying the product that can best detect a set of known malware files during a closed test. This type of testing is outdated and does not take into account how threats typically propagate today, or how computers are used. In this test, AV-Test.org utilized a testing methodology that is more real-world focused in which the samples are malicious URLs with associated malware files and allowed the solutions to block threats at their source (Exposure Layer), during download (Infection Layer) or on execution (Dynamic Layer). Also tested is the ability to source, analyze and protect against any samples that went undetected during the first round by re-testing 1 hour later.

Trend Micro outperforms all other vendors

Download report >

Total percentage of threats blocked by all layers:
Exposure, Infection, Dynamic

Overall Blocking Rate

Source: AV-Test.org | Note: Results are based on the T+60 minute results

And in the time to protect category, Trend Micro performed well.

Time to protect improvement percentage
% of previously unknown threats blocked at T = 60 minutes

Time to Protect

Source: AV-Test.org | NOTE: Time-to-protect improvement is the percentage of threats missed at T=0min that are subsequently prevented at T=60min. For example, with Trend Micro Worry-Free Business Security: At T=0min, 190 threats were prevented while 10 threats were missed. Of the 10 threats missed at T=0min, 1 was prevented at T=60min (1 of 10 equals 10%).

Why Trend Micro outperformed the competition

Trend Micro has outperformed the competition in a number of recent tests (See NSS Labs and Dennis Technology Labs) due to the Smart Protection Network™ infrastructure that powers products for consumer, SMB, Enterprise, Partner and SaaS customers but also powers our mobile, endpoint, server, messaging, gateway and SaaS solutions. This unique cloud-client architecture that uses Smart feedback to source new potential threats, patent-pending correlation to analyze multiple threat vectors (web, email, file) in real-time and in-the-cloud reputation databases to deliver protection wherever our customers connect, allows Trend Micro to react faster than other vendors against new threats. Our cloud-based protection network is the most evolved and comprehensive network in the industry blocking billions of email, web and file-based threats targeting our customers daily.

 

Trend Micro Consistency over Time

Traditional tests have focused on identifying the product that can best detect a set of known malware files during a closed test. This type of testing is outdated and does not take into account how threats typically propagate today, or how computers are used. As such, Trend Micro has been commissioning AV-Test.org monthly to utilize a testing methodology that is more real-world focused in which the samples are malicious URLs with associated malware files and allowed the solutions to block threats at their source (Exposure Layer), during download (Infection Layer) or upon execution (Dynamic Layer). Also tested was the ability to source, analyze and protect against any samples that went undetected during the first round by re-testing 1 hour later. Over time, it has become apparent that Trend Micro’s approach using the Smart Protection Network and multiple layers of protection has identified an ability to consistently block real-world threats more effectively than its competitors.

From the chart below, you see the overall results for each product tested (Enterprise Endpoint Solutions) since May 2010 through May 2011. Most vendors have wide fluctuations or lower numbers in their results which are due to a number of factors:

  1. Vendor is still focused on file-based protection which is difficult to keep updated with the number of new threats being released each hour.
  2. Vendor does not provide blocking of malicious URLs, which is the source of most infections today.
  3. Vendor does not have automated sourcing of unknown threats.
  4. Vendor does not have automatic correlation of new threat data in order to quickly identify unknown threats
  5. Vendor does not have cloud-based protection and still requires signature files to be delivered to each and every endpoint computer, thus delaying the time to protect.

Overall Results Total Percentage of all threats blocked at each layer:
Exposure, Infection and Dynamic
AV-Test Historical Results

Overall Result

Source: AV-Test.org | Note: Results are based on the T=0 minute results. Test results can vary over time and may vary with other configuration settings.

Why Trend Micro outperformed the competition

Trend Micro has outperformed the competition in a number of recent tests (See NSS Labs and Dennis Technology Labs) due to the Smart Protection Network™ infrastructure that powers products for consumer, SMB, Enterprise, Partner and SaaS customers but also powers our mobile, endpoint, server, messaging, gateway and SaaS solutions. This unique cloud-client architecture that uses Smart feedback to source new potential threats, patent-pending correlation to analyze multiple threat vectors (web, email, file) in real-time and in-the-cloud reputation databases to deliver protection wherever our customers connect, allows Trend Micro to react faster than other vendors against new threats. Our cloud-based protection network is the most evolved and comprehensive network in the industry blocking billions of email, web and file-based threats targeting our customers daily.

Archives