virustotal fun

I hacked a script to retrieve the virustotal results for the files mentioned in the paris database, and store the results in the paris database so I could query them. Unfortunately dionaea does not submit to virustotal.com (yet), therefore there are signatures missing for 'some' (75%) files. Afterwards I designed a queries to retrieve some stats about different things.

Conficker/Kido domination

As I was interested in the share of Conficker attacks, I decided to retrieve some numbers from the paris database.
As I don't know which files count as Conficker, I had to rely on av vendor signatures.

SELECT
	strftime('%Y-%m-%d', connection_timestamp, 'unixepoch') AS whenever,
	total,
	COUNT(*) AS hits,
	ROUND((CAST(COUNT(*)*100 AS FLOAT) / CAST(total AS FLOAT)),2) AS cut,
	av_result
FROM 
	downloads
	NATURAL JOIN connections
	LEFT OUTER JOIN avs ON ( av_md5_hash = download_md5_hash AND av_vendor = 'Kaspersky')
	JOIN (
		SELECT 
			COUNT(*) AS total,  
			strftime('%Y-%m-%d', x.connection_timestamp, 'unixepoch') AS pd 
		FROM 
			downloads AS d 
			NATURAL JOIN connections AS x 
		GROUP BY 
			strftime('%Y-%m-%d', x.connection_timestamp, 'unixepoch')
		) ON(pd = strftime('%Y-%m-%d', connection_timestamp, 'unixepoch'))
GROUP BY
	strftime('%Y-%m-%d', connection_timestamp, 'unixepoch'),
	av_result
ORDER BY
	strftime('%Y-%m-%d', connection_timestamp, 'unixepoch') ASC,
	COUNT(*) DESC,
	av_result ASC
day total hits share signature
2009-11-30 86877 75883 87.35 Net-Worm.Win32.Kido.ih
2009-11-30 86877 11118 12.8 (null)
2009-11-30 86877 349 0.4 Trojan.Win32.Buzus.cqjn
2009-11-30 86877 216 0.25 Net-Worm.Win32.Kolab.ffa
2009-11-30 86877 152 0.17 Trojan.Win32.Buzus.crgf
2009-11-30 86877 117 0.13 Trojan.Win32.Kreeper.aph
2009-11-30 86877 16 0.02 Net-Worm.Win32.Kido.dam.y
2009-11-30 86877 16 0.02 Trojan.Win32.Kreeper.aub
2009-11-30 86877 10 0.01 Trojan.Win32.Buzus.cqbm
2009-11-30 86877 8 0.01 -
2009-11-30 86877 3 0.0 Backdoor.Win32.Rbot.aftu
2009-12-01 106259 89644 84.36 Net-Worm.Win32.Kido.ih
2009-12-01 106259 16990 15.99 (null)
2009-12-01 106259 242 0.23 Net-Worm.Win32.Kolab.ffa
2009-12-01 106259 144 0.14 Net-Worm.Win32.Kido.dam.y
2009-12-01 106259 45 0.04 Trojan.Win32.Kreeper.aph
2009-12-01 106259 12 0.01 -
2009-12-01 106259 2 0.0 Backdoor.Win32.Nepoe.px
2009-12-01 106259 2 0.0 Trojan.Win32.Buzus.cqjn
2009-12-01 106259 1 0.0 Trojan.Win32.Buzus.crgf
2009-12-02 103490 87599 84.64 Net-Worm.Win32.Kido.ih
2009-12-02 103490 16644 16.08 (null)
2009-12-02 103490 101 0.1 Net-Worm.Win32.Kolab.ffa
2009-12-02 103490 48 0.05 Net-Worm.Win32.Kido.dam.y
2009-12-02 103490 25 0.02 -
2009-12-02 103490 3 0.0 Virus.Win32.Virut.av
2009-12-02 103490 1 0.0 Trojan.Win32.Kreeper.aub
2009-12-03 101270 87789 86.69 Net-Worm.Win32.Kido.ih
2009-12-03 101270 14721 14.54 (null)
2009-12-03 101270 137 0.14 Trojan.Win32.Buzus.crty
2009-12-03 101270 97 0.1 -
2009-12-03 101270 8 0.01 Trojan.Win32.Kreeper.aub
2009-12-03 101270 1 0.0 Net-Worm.Win32.Kido.dam.y
2009-12-04 105346 89665 85.11 Net-Worm.Win32.Kido.ih
2009-12-04 105346 14320 13.59 (null)
2009-12-04 105346 2407 2.28 Trojan.Win32.VB.yzp
2009-12-04 105346 189 0.18 Trojan.Win32.Buzus.csex
2009-12-04 105346 152 0.14 Trojan.Win32.Buzus.crty
2009-12-04 105346 38 0.04 -
2009-12-04 105346 37 0.04 Trojan.Win32.Kreeper.aub
2009-12-04 105346 33 0.03 Net-Worm.Win32.Kido.dam.y
2009-12-04 105346 4 0.0 Virus.Win32.Virut.av
2009-12-05 105608 88950 84.23 Net-Worm.Win32.Kido.ih
2009-12-05 105608 15364 14.55 (null)
2009-12-05 105608 2396 2.27 Trojan.Win32.Buzus.csex
2009-12-05 105608 159 0.15 Trojan.Win32.Buzus.crty
2009-12-05 105608 29 0.03 Net-Worm.Win32.Kido.dam.y
2009-12-05 105608 12 0.01 Trojan.Win32.Kreeper.aub
2009-12-05 105608 1 0.0 -
2009-12-06 99487 84142 84.58 Net-Worm.Win32.Kido.ih
2009-12-06 99487 14701 14.78 (null)
2009-12-06 99487 1647 1.66 Trojan.Win32.Buzus.csjr
2009-12-06 99487 64 0.06 Net-Worm.Win32.Kido.dam.y
2009-12-06 99487 55 0.06 Net-Worm.Win32.Lovesan.a
2009-12-06 99487 16 0.02 Trojan.Win32.Buzus.crty
2009-12-06 99487 11 0.01 -
2009-12-07 41181 36010 87.44 Net-Worm.Win32.Kido.ih
2009-12-07 41181 5132 12.46 (null)
2009-12-07 41181 385 0.93 -
2009-12-07 41181 64 0.16 Net-Worm.Win32.Kido.dam.y
2009-12-07 41181 1 0.0 Trojan.Win32.Buzus.crty

Conficker dominates, about 80% of all successful attacks on the honeypot are Conficker. Paris was a /23, 512 ips, ~88000 successful Conficker attacks within 24 hours is 61 attacks per minute, more than one successful attack a second(!).

The ”-” in the signature indicates Kaspersky did not provide a signature for the file, ”(null)” means virustotal had no match for the file.

virustotal unknown

First, retrieve the total number of unique files:

SELECT 
	COUNT(DISTINCT download_md5_hash) FROM downloads;
total
2064

2064 files in a week, good catch.

Then, the number of files unknown to virustotal:

SELECT COUNT(DISTINCT download_md5_hash)
FROM 
	downloads 
	LEFT OUTER JOIN avs ON (av_vendor = 'Kaspersky' AND av_md5_hash = download_md5_hash) 
WHERE
	av_result IS NULL
count
1624

1624 files unknown to virustotal, thats about 78% of the total.

Maybe the files are unknown as the download failed, and we received an broken sample? To verify we count the number of downloads for files unknown to virustotal.

SELECT 
	COUNT(download_md5_hash), 
	download_md5_hash,
	av_result
FROM 
	downloads 
	LEFT OUTER JOIN avs ON (av_vendor = 'Kaspersky' AND av_md5_hash = download_md5_hash) 
WHERE
	av_result IS NULL
GROUP BY 
	download_md5_hash 
ORDER BY 
	COUNT(download_md5_hash) DESC
	LIMIT 10
count md5sum signature
1382 6cd3a0ff50bd46b70ad3eaa6ab6e522a (null)
1268 61ddb216aea513c941b4e0ab7e7165b6 (null)
1257 f80fe83fdcb3650fb0a7af5a8e33f125 (null)
1140 6b54e187a3a6971ffe03e9aea5afcacc (null)
1131 0736ce6f0bd61a4d28dcfd22d5efca7a (null)
1070 e0d9893863d36b59ed8dff83e0566db8 (null)
1045 4d37c4497728bcbc5f5f8ffdd171f482 (null)
1039 d5d66bd48117c4c8091033650fb8fc85 (null)
1004 5a9ece8153de6e837887b40d8a2e0355 (null)
998 79e66c4ccbedaa99415ad85f75e5c006 (null)

So, files where we had 1382 successful downloads during a week are unknown to virustotal, definitely no single broken samples. I had to limit the query to the first 10, as the whole list of 1624 samples does not provide any new value.

blind spots

If virustotal has the file, but a av vendor does not detect it, we call it a blind spot.

SELECT 
	COUNT(av_vendor),
	av_vendor
FROM 
	(SELECT DISTINCT download_md5_hash  FROM downloads)
	JOIN avs AS o ON (o.av_md5_hash = download_md5_hash)  
WHERE
	o.av_result = '-'
GROUP BY 
	o.av_vendor
ORDER BY 
	COUNT(av_vendor) DESC
count (less is better) vendor
192 eSafe
123 nProtect
114 Comodo
74 ClamAV
70 PCTools
68 Fortinet
50 Authentium
46 F-Prot
42 Prevx
34 ViRobot
20 AhnLab-V3
20 TheHacker
18 VirusBuster
17 F-Secure
17 Ikarus
17 McAfee
16 Norman
15 Jiangmin
15 eTrust-Vet
14 Antiy-AVL
13 Microsoft
11 Rising
11 Sunbelt
10 McAfee+Artemis
10 Sophos
10 TrendMicro
10 VBA32
10 a-squared
9 K7AntiVirus
9 Panda
8 Symantec
7 AntiVir
7 McAfee-GW-Edition
6 AVG
6 Avast
6 BitDefender
6 CAT-QuickHeal
6 NOD32
5 GData
4 DrWeb
4 Kaspersky
2 Prevx1
1 Ewido
1 SecureWeb-Gateway

Given the fact we have only about 400 results from virustotal, 192 blind spots mean your av solution would detect 50% (!) from the malware the honeypot gatherd from the wild in a single week.

Kasperskys blind spots

Next point, how many blind spots has specific virus scanner, I choose Kaspersky as 4 blind spots is a good value to query, goind for 1 or 192 blind spots is pointless.

SELECT 
	COUNT(download_md5_hash), 
	download_md5_hash,
	av_result
FROM 
	downloads 
	LEFT OUTER JOIN avs ON (av_vendor = 'Kaspersky' AND av_md5_hash = download_md5_hash) 
WHERE
	av_result = '-' 
GROUP BY 
	download_md5_hash 
ORDER BY 
	COUNT(download_md5_hash) DESC
count md5sum signature
394 bb279e571735ac35d3de70b830f23b6f -
150 acc6dbf1d92baf4af234a6a9fc063e3f -
29 824ac57e222d1b09eeac551505e1cd72 -
4 979b3d197cf71be7f98c9d9e9acb61c0 -

From the 400 files virustotal knew, there are 4 where Kasperky did not provide a signature, maybe other av vendors got a name on it?

SELECT 
	download_md5_hash,
	o.av_vendor,
	o.av_result
FROM 
	downloads 
	JOIN avs AS k ON (k.av_vendor = 'Kaspersky' AND k.av_md5_hash = download_md5_hash)  
	JOIN avs AS o ON (o.av_md5_hash = download_md5_hash)  
WHERE
	k.av_result = '-' AND o.av_result != '-'
GROUP BY 
	download_md5_hash, o.av_vendor
md5sum signature vendor
824ac57e222d1b09eeac551505e1cd72 AVG BackDoor.Generic_c.BJV
824ac57e222d1b09eeac551505e1cd72 AntiVir TR/Crypt.XPACK.Gen
824ac57e222d1b09eeac551505e1cd72 Antiy-AVL Backdoor/Win32.IRCBot.gen
824ac57e222d1b09eeac551505e1cd72 BitDefender Trojan.Generic.2608372
824ac57e222d1b09eeac551505e1cd72 CAT-QuickHeal (Suspicious) - DNAScan
824ac57e222d1b09eeac551505e1cd72 ClamAV Trojan.Sdbot-3424
824ac57e222d1b09eeac551505e1cd72 Comodo UnclassifiedMalware
824ac57e222d1b09eeac551505e1cd72 DrWeb BackDoor.IRC.Sdbot.901
824ac57e222d1b09eeac551505e1cd72 F-Secure Backdoor.Bot.35586
824ac57e222d1b09eeac551505e1cd72 GData Trojan.Generic.2608372
824ac57e222d1b09eeac551505e1cd72 Ikarus Backdoor.IRC.Sdbot.901
824ac57e222d1b09eeac551505e1cd72 Jiangmin Backdoor/IRCBot.p
824ac57e222d1b09eeac551505e1cd72 McAfee potentially unwanted program Corrupt-07!824AC57E222D
824ac57e222d1b09eeac551505e1cd72 McAfee+Artemis potentially unwanted program Corrupt-07!824AC57E222D
824ac57e222d1b09eeac551505e1cd72 McAfee-GW-Edition Trojan.Crypt.XPACK.Gen
824ac57e222d1b09eeac551505e1cd72 Panda W32/RxBot.FL.worm
824ac57e222d1b09eeac551505e1cd72 Prevx Medium Risk Malware
824ac57e222d1b09eeac551505e1cd72 Symantec Suspicious.MH690.A
824ac57e222d1b09eeac551505e1cd72 VBA32 BackDoor.IRC.Sdbot.901
824ac57e222d1b09eeac551505e1cd72 a-squared Backdoor.IRC.Sdbot.901!IK
acc6dbf1d92baf4af234a6a9fc063e3f NOD32 a variant of Win32/Conficker.X
acc6dbf1d92baf4af234a6a9fc063e3f Symantec W32.Downadup
acc6dbf1d92baf4af234a6a9fc063e3f eSafe Suspicious File

Yes, some recognize the files Kaspersky does not, and as usual there are multiple names.

Seems like I'll have to extend the script to upload unknown files to virustotal. The plan is to offer the new 'avs' table as mission pack to the paris database, and publish the virustotal script.

Comments

1

Nice work as usual Markus.

Only word of caution I'd suggest (and likely something that you're already aware of) is that I've been caught out in the past claiming a binary as no or poor AV coverage based on VirusTotal only to find it's actions trigger every known AV protection once run in a live environment.

Definitely like the list of blindspots per vendor. Will be useful statistics when justifying purchases (or the avoidance of a purchase) in future.

Thanks for the work, Andrew Waite

Andrew Waite
2009/12/14 17:24
2

I updated dionaea to the latest version and find several new modules is embedded. I just enabled the virustotal and change the API to my key, is it enough or I have to update the logsqlite as well?

honeybirdhk
2010/09/21 11:34
3

The data virustotal returns is not stored via logsql yet.

Markus
2010/09/22 13:52
4

Good news! Virustotal API is great! :D

Jeroenz0r
2010/11/24 08:55
5

I know, but this post predates the api and the dionaea code to use it, but you are right, I never posted an update …

Markus
2010/11/24 22:13


2009/12/14/virustotal_fun.txt · Last modified: 2010/06/15 11:49 by common
chimeric.de = chi`s home Creative Commons License Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0