carniwwwhore is meant to be a webinterface for dionaea's postgres database which is fed over xmpp. To start with, you can convert your current logsql sqlite database to postgres, instructions are included. carniwwwhore is written in python2.x using django. If you experience problems following the guide, post a comment with your solution, or at least a good description, so I can update the instructions.

A common problem seems to be understanding the overall setup, so I used asciiflow to brighten the darkness. created using asciiflow and ditaa.

This post relates to the 'webinterface' box in the diagram, refer to these 'resources' for the other boxes:

• XMPP Server
• http://dionaea.carnivore.it/#pg_backend
• pg_schema
• sensor 1&2

• python
  • python-django >= 1.2.3
  • python-psycopg2
  • python-pygame
  • python-numpy
  • python-pychart
  • python-pythonmagick
• postgres database >= 8.3
• data

aptitude install python-django python-psycopg2 python-numpy python-pygame python-pychart python-pythonmagick

If your distribution lacks a recent django version, try installing a more recent package from your distributions more recent version. Simply grab the package and install using dpkg -i python-django_1.2.3-*_all.deb. Worked fine for me.

For debian lenny, you are lost, as the python version (2.5) is too old.

Install postgres

apt-get install postgresql

create a database user 'xmpp'

sudo -u postgres createuser --no-createdb --encrypted --login --pwprompt --no-createrole --no-superuser xmpp

Allow the database user local login (your config path may differ):

local   xmpp        xmpp                              md5 # <- add this line
local   all         all                               ident

/etc/postgresql/8.4/main/pg_hba.conf

Restart postgres to have the changes taking effekt

sudo /etc/init.d/postgresql restart

Create the database xmpp

sudo -u postgres createdb --owner xmpp xmpp

Create the tables - this requires the pg_schema.sql from a dionaea checkout

psql -U xmpp xmpp < PATH/dionaea/modules/python/util/xmpp/pg_schema.sql

For a start it is possible (and recommended) to convert the logsql database to postgres, this requires a python3 interpreter and py-postgresql which is python3 postgres binding as well as the logsql2postgres script from dionaea/modules/python/util/logsql2postgres.py.

If you lack a sqlite database, you can grab the berlin database from the ftp server to start with some data. Be aware you'll miss some data when using the berlin tables, for example the virustotal database does not even exist in the dump, as it was added later on, and you'll have to run dionaea on the berlin database, so dionaea can update the database to current format and apply some fixes.

It is recommended to copy the database and import the copy, else the database may change during the operation, and get inconsistent. To copy the db, stop dionaea, copy /opt/dionaea/var/dionaea/logsql.sqlite to /tmp and use the copy.

If your distributions lacks, you can refer to dionaeas guide how to install it.

I assume you have python3 installed in /opt/dionaea/ …

wget http://python.projects.postgresql.org/files/py-postgresql-1.0.2.tar.gz
tar zxvf py-postgresql-1.0.2.tar.gz
cd py-postgresql-1.0.2/
sudo /opt/dionaea/bin/python3 setup.py install

Now we can shovel the data from the logsql.sqlite database to the postgres database, be aware the logsql2postgres script does not allow incremental database updates as it will *clean* all data in the tables. You can even use a remote postgres database etc, just specify the host/username.

time dionaea/modules/python/util/logsql2postgres.py -s localhost -d xmpp -u xmpp -p MYPASSWORD -f /tmp/logsql.sqlite connections dcerpcservices dcerpcrequests downloads emu_profiles p0fs dcerpcserviceops dcerpcbinds offers emu_services mssql_fingerprints logins mssql_commands virustotals virustotalscans

Do not forget to adjust your password in the command, I know it is rather bad having a password visible in the processes list, if you object enough to create a fix, send me a patch.

Depending on the size of your database, this command will take a while, starting with some minutes …

You'll love it, as it requires you to setup an xmpp server, and link dionaea to this xmpp service, so you can connect a backend into this xmpp service which writes the data to the database.
For a guide how to do so, refer to the xmpp server docs.
Be aware the data received by xmpp is only a subset of the data received by logsql.
Adding the missing bits to logxmpp and pg_backend is a .

Now we got all requirements fulfilled and can start installing carniwwwhore itself.

mkdir /opt
cd /opt
git clone git://git.carnivore.it/carniwwwhore.git

Adjust your database settings in settings.py

cp settings.py-example settings.py
editor settings.py
# adjust
# DATABASES = {
#	'default': {
#		'ENGINE': 'django.db.backends.postgresql_psycopg2',
#		'NAME': 'xmpp',
#		'USER' : 'xmpp',
#		'PASSWORD' : 'test',
#	}

Start the django webserver:

cd /opt/carniwwwhore
python manage.py runserver

point your browser to http://127.0.0.1:8000/dionaea

firefox http://127.0.0.1:8000/dionaea

The 'ore comes with a django.wsgi file, which can be used to run it from apache.

An example Apache config file may look like this:

<VirtualHost *:80>
	ServerName HOSTNAME:80
	ServerAdmin webmaster@localhost
 
	Alias /robots.txt /opt/carniwwwhore/template/v2/static/robots.txt
	Alias /favicon.ico /opt/carniwwwhore/template/v2/static/favicon.ico
	AliasMatch ^/static/(.*) /opt/carniwwwhore/template/v2/static/$1
 
	<Directory /opt/carniwwwhore/template/v2/static>
		Order deny,allow
		Allow from all
	</Directory>
 
	DocumentRoot /var/www
 
	WSGIScriptAlias / /opt/carniwwwhore/django.wsgi
 
	ErrorLog ${APACHE_LOG_DIR}/HOSTNAME-error.log
 
	# Possible values include: debug, info, notice, warn, error, crit,
	# alert, emerg.
	LogLevel warn
 
	CustomLog ${APACHE_LOG_DIR}/HOSTNAME-access.log combined
</VirtualHost>

/etc/apache2/sites-enabled/HOSTNAME

Activate the new virtual host and restart Apache:

a2ensite HOSTNAME
/etc/init.d/apache2 restart

Running from Apache it is recommended to:

• disable DEBUG in settings.py
• enable sending errors per mail (very useful)
  • add your email address to ADMINS (settings.py)
  • set SERVER_EMAIL to the address the server should use to send mail

# Django settings for carniwwwhore project.
 
#DEBUG = True
DEBUG = False
TEMPLATE_DEBUG = DEBUG
 
ADMINS = (
    # ('Your Name', 'your_email@domain.com'),
	('webmaster','webmaster@HOSTNAME'),
)
 
SERVER_EMAIL = 'ore@HOST'

example for changes to settings.py

In case of 500 (internal server error) django will send a email to all addresses listed as ADMIN then.

NOTE You'll have to restart Apache for config changes and updates.

Problems :

• IOError: [Errno 13] Permission denied: '/dev/stdout'
  • change this to log to /dev/null or something instead

As I'm totally aware it looks ugly, there are no screenshots.
I'll post screenshots, once there is a better presentation of the data, starting with valid html.

I really like the overview, as it is reasonable fast, and I think it provides a good insight in whats going on by comparing current data to previous data. You can apply filters and 'zoom' in the aggregated data from the years overview to the hour overview, to spot peaks and switch to the dionaea/attacks overview for a list of attacks from the hour view.

List attacks with filters on all kinds of details.

Render all details for a single attack.

Lists the downloads, can be grouped by timeslot and filtered by md5hash, neat to spot for how long a given sample is active.

Lists all virustotal details we know about the malware received.

Given the current look and feel, I'd really appreciate if somebody would come up with some other templates, even if your templates are just proper html, this is already an improvement to the current templates.
If you are interested in contributing to carniwwhore, just send me your ssh pubkey accompanied by your first patchset, and I'll hook you up with git access, you can have your own tree or commit to master.

If you are new to django, like I am, read up on django by following their introduction:

• Part 1
• Part 2
• Part 3
• Part 4

If you want to improve the look&feel, you can do so easily by creating your own templates. The default templates are /opt/carniwwwhore/templates/dionaea/hateml/, if you want to create your own you can create a new directory copy over the ugly templates modify them:

cd /opt/carniwwwhore/
git cp templates/dionaea/hateml templates/dionaea/lovehtml/

Afterwards change the template path in settings.py to use your templates instead:

TEMPLATE_DIRS = (
    '/opt/carniwwwhore/template/dionaea/lovehtml'

Even though all views need love, the downloads view needs even more love, if you want to get into writing views, improving the current views is a good start.