=========================================
File 8PROBLMS.TXT
List of problems experienced during test:
=========================================
Formatted with non-proportional font (Courier)


Content of this file:
---------------------
    1. Introduction: General Problems 
       1.1 Problems likely related to FindFirst/FindNext anomaly
    2. List of benevolently behaving AV products
    3. Problems of AV products observed during tests


1. Introduction: General Problems:
----------------------------------
For automatic tests on large viral databases, and for automatic 
processing of large scanner log files, a set of test conditions 
is prerequisite for scanners to participate in a VTC test (see: 
4TESTCON.TXT).

In many cases, serious problems were observed during some tests. 
DOS scanners were either not suitably running under SIMBOOT and 
crashed, or problems appeared with the (rather large) file virus 
database. In some cases, scanners crashed upon detecting some 
specific virus; in few cases, "manual" operation instead of 
automatic (batch) operation helped solving some of these problems. 
Such curative action was also applied when possible in cases where 
log files were inadequate (e.g.needing manual operation for export).

With growing velocity of processors, DOS scanners (running without 
any problem on INTEL 386 and 486) growingly crash on Pentium II/III
systems faster than 250 MHz. Another general problem with DOS scanners 
is related to counters for files and viruses which often seem to be 
designed as integers, so they start after 65,536 with 0.

During preparation and test, we again experienced a serious problem 
reported in VTC Test "2001-04", according to which management of
large sets of directories in FAT and NTFS may not reliably work. Both
when attempting to move large parts of our file virus database, as 
when some scanner proceeded scanning subsequent viral directories, 
we found that several directories were not moved or touched. This
effect seems to happen stochastically, such that subsequent attempts
gave different results. Concerning omitted (=unscanned) directories,
we overcame this "dysfunctional" behaviour of FAT and NTFS by repeat-
ing scanning so long until the number of scanned files agreed with 
the (known) number of directories in testbeds. Overcoming this  
problem was extremely time-consuming, and this was a reason for 
delaying publication of results. 

In cases where scanners crashed during detection test upon the rather
large file virus database, tests were performed in several runs on
partitions (essentially on directories with same first letters of
names). In most cases (apart those reported below), these tests
were completed, and resulting files were joined and evaluated.

Finally, with growing testbeds, test protocols produced by scanners
grow equally. When processing such protocols, we need meanwhile up
to 6 GByte of disk space, and our evaluation scripts (in AWK) 
become more complex. Under these conditions, we also suffered from
an evident bug in the AWK processor which inhibited proper evaluation
and required additional quality assurance (including time and efforts). 

1.1 Problems very likely related to FindFirst/FindNext anomaly:
---------------------------------------------------------------
In several cases, scanners finished a first scan although they
had not touched all directories with infected objects. In such
a case, a postscan was started adressing only those untouched
objects; a second postscan was started when again objects were
observed untouched, but after the 2nd postscan, no more scan
was started. This behaviour may originate from a reported anomaly
in the behaviour of FindFirst/FindNext (those routines are used
to handle objects in directory trees) which has not been cured
so far by Microsoft.

In the "problems list", these postscans are ked as "minor problems":
different from crashes, postscans "only" required time for running
the tests and evaluating test protocols (which significantly delayed
results).

2. List of benevolently behaving AV products:
---------------------------------------------
In general, few scanners could be tested without any problem. When 
considering the large number of postscans, few products had only
minor problems and are regarded as "relatively benevolent".
 
Such "benevolent behaviour" can be reported only for a minority
of DOS scanners:
    ==========================================
    Out of 12 DOS scanners, 2 had NO problems:
    ------------------------------------------
               AVG  and  FPR
    ==========================================
    
In comparison, W-98 scanners were significantly less stable:
    ============================================
    Out of 23 W-98 scanners, 2 had only no or minor problems:
    ---------------------------------------------------------
      - executed with no problem:     (1) AT5
      - executed with minor problems: (1) PAV 
    ============================================

Concerning W-NT related scanners: 
    ==================================================
    Out of 18 W-NT scanners, the following 6 products
    -------------------------------------------------- 
      - executed with no problem:     (1) AVG
      - executed with minor problems: (5) AVP, CMD, 
                                      FPR, FPW and PAV
    ==================================================

Concerning Windows-2000 based scanners:
    	==============================================
	Out of 17 W-2000 scanners, the following 13 
		products had NO problems:
	----------------------------------------------
       - executed with no problem:     
                                (4) AVG, CMD, FPR, PAV 
       - executed with minor problems: 
                           (5) AV3, AVK, AVP, FPW, NAV
    	==============================================

Finally, concerning Linux-based scanners (tested for the first time):
    -------------------------------------------
    Out of 5 Linux scanners, 3 had no problems:
                CMD, RAV and SCN
    -------------------------------------------

Concerning overall stability:
	***************************************************************
	the following 3 products behaved with least problems (no crash,
	"only postscans"):                                CMD, FPW, RAV
        ***************************************************************



3. Problems of AV products observed during tests:
-------------------------------------------------
Concerning postscans (either due to untouched objects - see FF/FN
anomaly - or due to crashes of a product), the following list
summarizes those products where at least 1 postscan was initialised:

	W-NT: 	EXOT	 AVP, PRO
		MACRO	 SCN
		POLY	 SCN(2x), VSP(2x), FPW
		VKIT	 VSP, FPR(2x)
		FILE	 CMD, FPR, FPW, FSE(2x), NVC, SCN, VSP(2x)
		FILEMAL  AVK(2x), NVC(2x), VSP(2x)
		FILEPACK AVP
		MACPACK  AVP, FSE, PRO, SCN

	W-98: 	FILE	 ADO, AV3, AVG(2x), CLE, CMD, DRW, DSE, FPR, FPW, 
                         INO(2), MR2(2x), NAV, NVC(2x), PRO, SCN(2x), VSP
		FILEMAL	 ADO, ATR, AV3(2x), AVK(2x), AVP, DSE(2x), FPW(2x), 
                         MR2(2x), NAV(2x), NVC, PAV, PER, SCN(2x)
		FILEPACK ADO, FPR, FSE, PER, QHL, RAR
		MACRO	 ADO, AV3, AVK, AVP, CLE, DSE, FPW, INO, MR2(2x), 
			 NAV, NVC(2x), PER, PRO, QHL, SCN, VSP
		MACROITW AV3, FSE
		MACROMAL ADO
		MACPACK	 ADO(2x), AVP, FSE, INO, NAV, PAV(2x), QHL
		POLY	 DRW, MR2, SCN, VSP
		SCRIPT	 ADO, AV3, FSE
		SCRIPTITW ADO
		VKIT	 FSE

	W-2k: 	FILE	 AV3, CMD, FPR, FSE, INO, NAV, NVC(2x), PAV, 
			 PRO(2x), SCN(2x), VSP
		FILEMAL	 AVK, AVP, FPW, FSE, INO, NVC(2x), PER, PRO, 
			 RAV(2x), VSP
		MACRO	 AV3, AVK, INO, NAV, NVC, PER, PRO, SCN(2x), VSP
		POLY	 SCN, VSP
		VKIT	 SCN


The following list reports specific problems observed for products
as indicated ("spoon-feeding" means that scanner was restarted on each
subsequent directory when a crash was experienced):


    ANT     Product couldnot be installed and tested, as CD was 
	    physically broken (CD arrived significantly after deadline)


    ATD:    W-98:  Date reset (Jan.1, 2001)
                   Saving logs requires clicking with right mouse button
                           on "infected list"
	    
    ATR:    W-98:  no problems
	    
	    
    AVA     DOS:   Crashed three times on BOOT-ZOO; excluded then from this 
                           part of the test (no results)
		   Crashed three times on BOOT-ITW; excluded then from this 
                           part of the test (no results)
    AV3     W-98:  minor problems (postscans) 
	    W-NT:  no problems
	    W-2k:  minor problems (postscans)
	    

    AVG     DOS:   no problems
	    W-98:  Crashed at u:\23w\mronon\c\otpyrc\lld\dlb_000_.dll,
                   Diagnosis: the instruction at 0x00440115 referenced 
                        memory at 0x00851000. the memory could not be read.
	    W-NT:  no problems
	    W-2k:  no problems
	    

    AVK     DOS:   AVK has a maximum size for its report files of 512 Kb. 
                           Several reports were uncomplete because of this 
                           limitation. This forced several postscans.
                   Was unable to scan several files; for this product, these
                           files were nonexistent.
	    W-98:  Hangs after long scans (poly, Vkit, file viruses)
	    W-NT:  minor problems (postscans)
	    W-2k:  Crashed once when scanning File viruses
	    

    AVP     DOS:   Crashed once during Boot-tests
                   Problems when scanning CAB-Packed files: sometimes crashed 
                           with exception #0e, error code 0004
                   Was unable to scan several files; for this product, these
                           files were nonexistent.
	    W-98:  minor problems (postscans)
	    W-NT:  minor problems (postscans)
	    W-2k:  minor problems (postscans)
            Linux: Product crashed when processing several CABbed viral objects
                   such as: /k:packmac/W97M/MARKER/X/CAB.CAB
			    /k:packmac/W97M/MELISSA/A@MM/CAB.CAB
			    /k:packmac/W97M/MELISSA/AL@MM/CAB.CAB
			    /k:packmac/W97M/MELISSA/I@MM/CAB.CAB



    CLE:    W-98: Product always hangs when scanning File-Malware at
		  T:\NAJORT\MRONON\S\NEVESBUS\ROODKCAB\312v\EXA_002_.EXE
	    

    CMD:    W-98:  no problems
	    W-NT:  minor problems (postscans)
	    W-2k:  no problems
	    Linux: no problems


    DSE:    W-98:  minor problem (postscans)
	    	    
    
    DRW:    DOS:   Crashed three times on BOOT-ZOO; excluded from
                       this part of the test (no results)
		   Crashed twice on u:\sod\mronon\a\aesa\4443\exb_000_.exe
		   Crashed on u:\sod\mronon\w\59w\ortsib\neg\exb_000_.exe
	           Crashed on u:\sod\mronon\h\cllh\0967\exa_000_.exe; 
                       excluded from this part of the test (no results)
		   Crashed on t:\ksir_ces\mronon\b\roodkcab\enilovla.v\bab_001_.bat
		   Crashed three times during Malware-test; excluded from 
                       this part of the test (no results)
	    W-98:  Crashed more than 3 times at
		       T:\NAJORT\MRONON\B\G-ROODC.AB\RUS\EXB_001_.EXE                  
	    W-NT:  Scanner attempted to clean any virus; this failed as all 
                       objects were right-protected. 


    FPR:    DOS:    no problems
	    W-98:   reports were truncated for larger testbeds on R: (Macrovir), 
                         T: (Filemal), V: (Poly), W: (vkit) and U: (Filevir);
                         problems were solved in several postscans.
	    W-NT:   minor problems (postscans)
	    W-2k:   minor problems (postscans)
	    
 
    FPW     W-98:   minor problems (postscans)
	    W-NT:   minor problems (postscans)
	    W-2k:   minor problems (postscans)
	    

    FSE     W-98:  serious problems/several postscans
	    W-NT:  Crashed with blue screen when scanning U:\ at file 117;
                         when scan was repeated with memory dump enabled,
                         no crash observed
	    W-2k:  Crashed 3 times when scanning File viruses; deinstallation 
                         impossible
	    Linux: Problems with installation script (no input control) caused
                   erroneous diagnoses. Otherwise: no problems.


    INO     DOS:   Crashed on t:\najort\mronon\b\db-roodk.cab\rvs\exb_001_.exe
                   Crashed during Malware-Tests 
                   Displayed "ueberlauf bei Division" =overflow at division after scanning
	    W-98:  minor problems (postscans)
	    W-NT:  minor problem (postscans)
	    W-2k:  minor problems (postscans)


    MR2:    DOS:  Crashed two times on v:\oneh3544\onea0000.e\7\one9799.exe
                  Crashed while scanning folder v:\oneh3544\onea0000.e\8\
                  Crashed three times on BOOT-ZOO; excluded then from this 
                           part of the test (no results)
		  Crashed three times on BOOT-ITW; excluded then from this 
                           part of the test (no results)
                  Crashed three times on u:\sod\mron\s\kwauqs\258\exa_003_.exe
                  Crashed three times scanning file-zoo during after-test; 
                          excluded from this part of the test (no results)
	    W-98: Incomplete logs when scanning R,T,V,W,U; problems were 
                          solved in several postscans.
	   

    NAV     DOS:  Crashed once during BOOT-test
                  Didnot report all files checked (no OK-Message). Several post-tests 
                          were made, with the last one starting the scanner dirctly 
                          on every folder. Even after this (rather exhaustive and
                          time consuming) test, it was impossible to determine whether 
                          missed files where not identified or completely missed.
	    W-98:  Crashed once when scanning U: (filevir), but 2nd run was successfull
	    W-NT:  minor problems (postscans)
	    W-2k:  minor problems (postscans)	    


    NVC:    DOS:   Crashed once during FILE-tests
                   Crashed while unpacking u:\sod\mronon\z\reppiz\9772\a\exa_003_.exe
                   Crashed while scanning FILE-ZOO
                   Crashed while scanning u:\59W; excluded from 
                        this part of the test (no results)
                   Crashed three times with total system reboot on file-malware; 
                        excluded from this part of the test (no results)
                   Crashed with memory crash
                   Crashed twice after scanning k:\o97\halfcros\a_doc_\arj.arj 
                        with message: "Unpacking Zip.Zip"
		   Crashed after scanning k:\wm\cap\a\arj.arj
                        with message: "Unpacking Zip.Zip"; excluded from 
                        this part of the test (no results)
                   Crashed 3 times after scanning 
                        L:\file\23w\mronon\f\evolnuf\9904\arj.arj 
                        with message: "Unpacking Zip.Zip"; excluded from 
                        this part of the test (no results) 
	    W-98:  seriuous problems (postscans)
	    W-NT:  no problems
	    W-2k:  Crashed once when scanning File viruses
	    

    PAV:    DOS:   PAV crashed twice while loading antiviral databases while 
                   scanning Boot; once, it created a 1,5 GB report file
	    W-98:  minor problems (postscans)
	    W-NT:  minor problems (postscans)
	    W-2k:  minor problems (postscans)
	    

    PER:    W-98:  Crashed when scanning U:\sod\mronon\t\laivirt\121\b\c0a_000_.com
	    W-NT:  Crashed when scanning U:\ at File 128027 
	    W-2k:  Crashed 3times when scanning File viruses (after 128,027 files)
	    

    PRO     W-98:  Hangs upon scanning W: (vkit) and U: (Filevir)
	    W-NT:  minor problems (postscans)
	    W-2k:  minor problems (postscans)
	    

    QHL:    W-98:  Many crashes for several testbeds observed: FileVir, FileMal,
                           ScriptVir, ScriptVir-ITW. No results where more than
                           3 crashes.
		   Remark: product detected macro viruses only when packed.
	    
	    
    RAV     W-98:  minor problems (postscans)
	    W-NT:  no problems
	    W-2k:  minor problems (postscans)
	    Linux: no problems

    SCN     DOS:   Crashed once (during postscan) on 
                         u:\sod\mronon\s\f_gnimae.rcs\729\coa_002_.com
	    W-98:  Incomplete logs when scanning R,T,V,W,U; problems were 
                          solved in several postscans.
	    W-NT:  minor problems (postscans)
	    W-2k:  minor problems (postscans)
	    Linux: no problems
	     

    VSP     DOS:   Crashed on v:\maltese\ma1000.exe\0\7\ma0762.exe
		   Crashed on v:\oneh3544\one2500.e\8\one5892.exe
		   Crashed on v:\oneh3544\onea0000.e\8\one9854.exe; excluded 
                           from poly-test after triple crash (no results)
                   Crashed three times on BOOT-ZOO; excluded then from this 
                           part of the test (no results)
		   Crashed three times on BOOT-ITW; excluded then from this 
                           part of the test (no results)
                   Crashed on u:\sod\mronon\t\laivirt\821\a\coa_000_.com
                   Crashed during FILE-ZOO test
                   Crashed on u:\sod\mronon\t\laivirt\82\_nwonknu\coa_000_.com; 
                           excluded from this part of the test (no results)
	    W-98:  Incomplete logs when scanning R,T,V,W,U; problems were 
                          solved in several postscans.
                   Comment: when scanned from a batch file, every batch had to
                          started with CRLF; with VTC testbed complexity, this
                          required about 65 explicit CRLFs on U:\file :-)
	    W-NT:  minor problems (postscans)
	    W-2k:  minor problems (postscans)



	    
