[sanesecurity] Re: Long DB refresh times

  • From: "Paul Enlund" <paul@xxxxxxxxxxxxx>
  • To: sanesecurity@xxxxxxxxxxxxx
  • Date: Wed, 25 Apr 2012 12:21:01 +0100

>
>> I have just done some tests and bofhland_malware_URL.ndb increases my
>> database reload time by 400%.
>> Total number of sigs. loaded is not a factor in the reload time increase
>>
>
>
> Hi Paul,
>
> I think I might have made a little progress, after doing a few tests on
> signature variations...
>
> Can do do the following test for me...
>
> 1. Load db's as normal.. make a note of the time to load.
> 2. sed -i "s/:687474703A2F2F/:2F2F/g" bofhland_malware_URL.ndb
> 3. reload and take a note of the time.
>
> Any improvement?
>
> Simple test for me using clamscan and one database only:
>
> Before:
>
> clamscan --database=bofhland_malware_URL.ndb test.eml
>
> test.eml: OK
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 31999
> Engine version: 0.97.3
> Scanned directories: 0
> Scanned files: 1
> Infected files: 0
> Data scanned: 0.01 MB
> Data read: 0.00 MB (ratio 2.00:1)
> Time: 42.625 sec (0 m 42 s)
>
> After:
>
> clamscan --database=bofhland_malware_URL.ndb test.eml
>
> test.eml: OK
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 31999
> Engine version: 0.97.3
> Scanned directories: 0
> Scanned files: 1
> Infected files: 0
> Data scanned: 0.01 MB
> Data read: 0.00 MB (ratio 2.00:1)
> Time: 2.031 sec (0 m 2 s)
>
> So, from 42s down to 2s
>
> Cheers,
>
> Steve
> Sanesecurity
>
>
>

Ist sed'd db as per instructions.
uac-vm1:/home/paule# clamscan --database=bofhland_malware_URL.ndb 
/etc/hostname
/etc/hostname: OK

----------- SCAN SUMMARY -----------
Known viruses: 31999
Engine version: 0.97.3
Scanned directories: 0
Scanned files: 1
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 0.918 sec (0 m 0 s)

With original db

uac-vm1:/home/paule# clamscan
--database=/usr/unofficial-dbs/ss-dbs/bofhland_mal
ware_URL.ndb  /etc/hostname
/etc/hostname: OK

----------- SCAN SUMMARY -----------
Known viruses: 31999
Engine version: 0.97.3
Scanned directories: 0
Scanned files: 1
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 22.573 sec (0 m 22 s)



clamd load times
Before sed
Wed Apr 25 11:44:04 2012 -> Reading databases from /var/lib/clamav/
Wed Apr 25 11:45:28 2012 -> Database correctly reloaded (1901488 signatures)
After sed
Wed Apr 25 11:45:51 2012 -> Reading databases from /var/lib/clamav/
Wed Apr 25 11:46:13 2012 -> Database correctly reloaded (1901488 signatures)


PE


Other related posts: