There has been lately a lot of attention given to proactive antivirus defense methods, which allow the antivirus to resist types and modifications of malware not yet known. Such a direction of development is the most perspective on the market, and almost every developer tries to highlight that it is his/her proactive defense that is the best. Moreover, attempts are made to contrast in a way the newest proactive technologies with the old classic reactive ones, which are based on the signature methods of malware detection and demand permanent and often upgrading of the antivirus databases

Proactive Detection Test

In this chapter we intend to inform you about the results of independent Antivirus software tests, prepared and organized by the experts of the project Anti-Malware.ru

The concept itself of the proactive defense is certainly attractive – the virus does not exist yet, but the defense against it is there. But there is a question: how effective are these technologies?

 We will mention right away that the proactive technology is a fairly loose notion. It comprises numerous directions and components; that is why, to cover all of them within a test is not possible.

In this test we will compare only the heuristic components of antivirus protection (heuristic + generic detection, i.e. extended signatures), without taking into account the analysis of the system events (Behavioral Heuristic Analyzer).

The results of these tests give the answers to the questions: “How efficient is heuristic analyze? In which antivirus this component works the best?”

As appendix to the test, there has been undertaken a final measurement of the detection level on the collection of anti-virus programs with updated virus databases  a week after the essential test was finished.

 As a result, the unknown malware detection quality has been fixed, as well as work efficiency of the classic signature methods of each antivirus as supplement to the heuristic.

Results of the Proactive Protection test (08/12/2007)

Avira AntiVir Personal Edition Premium 7.0 (71%)  BitDefender Antivirus 2008 (65%)	Gold  Proactive Protection Award
Eset Nod32 Anti-Virus 3.0 (59%)  Dr.Web 4.44 (57%)  Sophos Anti-Virus 7.0 (56%)  Avast! Professional Edition 4.7 (52%)  VBA32 Antivirus 3.12 (48%)  Kaspersky Anti-Virus 7.0 (45%)  McAfee VirusScan Plus 2008 (43%)	Silver Proactive Protection Award
Symantec Anti-Virus 2008 (38%)  AVG Anti-Virus Professional Edition 7.5 (37%)  F-Secure Anti-Virus 2008 (36%)  Trend Micro Antivirus + Antispyware 2008 (30%)  Panda Antivirus 2008 (20%)	Bronze Proactive Protection Award
Agnitum Outpost Security Suite 2008 (12%)	TEST FAILED

Proactive antivirus protection testing methodology

Test was performed on a VMware GSX Server dedicated for that purpose. Separate Virtual PC with Microsoft Windows XP SP2 was cloned for every "experimental" antivirus product.

The following Anti-Virus products were tested in this comparative:

1. Agnitum Outpost Security Suite 2008
2. Avast! Professional Edition 4.7
3. AVG Anti-Virus Professional Edition 7.5
4. Avira AntiVir Personal Edition Premium 7.0
5. BitDefender Antivirus 2008
6. Dr.Web 4.44
7. Eset Nod32 Anti-Virus 3.0
8. F-Secure Anti-Virus 2008
9. Kaspersky Anti-Virus 7.0
10. McAfee VirusScan Plus 2008
11. Panda Antivirus 2008
12. Sophos Anti-Virus 7.0
13. Symantec Anti-Virus 2008
14. Trend Micro Antivirus plus Antispyware 2008
15. VBA32 Antivirus 3.12

During installation procedure all the recommended program’s events (restarting, upgrading, etc) were performed. All the defense components were activated, in case this action has not been supported after installation automatically.

Heuristic analyzer verification method

After the preparation of the "testing stand", special conditions were created to verify heuristic analyzer work efficiency. One of them was to deactivate /disconnect the upgrading function; thus, the antivirus databases were frozen at the beginning of the test. 

Wild models of malicious software were chosen from “the wild nature” (gateways, entrances/receptions in private collections) and replenished  our experimental collection two weeks after the antivirus databases were frozen.

The ITW-models (ITW - "in the wild") accumulated during six months until the test began. There was a selection of malware not yet known to the antivirus at the moment when upgrading was stopped.

What is important is the following: the two-week-gap between the antivirus databases freeze and the malware selection was intended on purpose to minimize the possibility of falling into the collection of the models that are known to any antivirus.

As a result of all these actions the defense efficiency of the classical signature components was nonexistent. Thus, any detection of an unknown model while being scanned on request could take place only with the proactive heuristic component, as was intended.

Scanning methods

Scan on request took place with all options on: maximum heuristic analyzer scan level, "scan all files" option enabled, all malware types were included, as well as potentially dangerous programs detection option activated.

As the appendix, a week after the main test all the antivirus programs were updated and test was repeated in order to determine the effectiveness of classical signature-verification method. Consequently, the work efficiency of both methods were verified and recorded.

Testing environment creation sequence

1. Anti-Virus clear installation (just after clear OS installation)

2. System reboot

3. Successful installation and modules operability verification

4. Anti-Virus database update

5. System reboot

6. Anti-Virus upgrade function deactivation, system disconnection from the Internet

7. Virtual machine image was saved

8. Virtual machine was shut down for six weeks

9. Collection of malicious software for the test

Testing procedure

1. Virtual machine start up

2. Malicious software database on-demand-scan (with "automatically delete infected object" option enabled)

3. Number of found objects were determined

4. Anti-Virus update

5. repeated on-demand malware database scan

6. Number of remaining objects were determined

Detailed results of proactive antivirus protection test

Test was performed on a VMware GSX Server dedicated for that purpose during the period from October 21, 2007 to December 8, 2007 strictly in compliance with described above methodology. Separate Virtual PC with Microsoft Windows XP SP2 was cloned for every "experimental" antivirus product.

Graph of Heuristic Analyzer test results:

Change in time of antivirus protection effectiveness

In compliance with testing methodology, monthly collection of  "fresh" malicious software samples (collected in the period from November 5 to December 2) was used to test how work effectiveness of different heuristic analyzers varies with time. For that purpose we divided whole collection into 4 equal parts - one part for each "testing week".

Thus, on the next graph  we can observe data obtained from the heuristic analyzer test made on weekly virus collections.

Test results separated by weeks:

Conclusion

As a result, heuristic component of Avira AntiVir Personal Edition Premium 7.0, BitDefender Antivirus 2008, Kaspersky Anti-Virus 7.0 and Dr.Web 4.44  shows the most effective and stable work. Heuristic component of Eset Nod32 is effective but very unstable while varying in time; besides, it lost its effectiveness considerably at the end of testing period.

Author of original article: Сергей Ильин