I've had some problems with ssl lately, here is what I found to be the problem/solution.

The problem is pretty easy, inability to access https services, mwanalysis.org may serve as an example here.

I was able to reproduce the problem using python(3.2):

import urllib.request
url="https://mwanalysis.org"
a = urllib.request.urlopen(url)

Traceback (most recent call last):
  File "/opt/dionaea/lib/python3.2/urllib/request.py", line 1122, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/opt/dionaea/lib/python3.2/http/client.py", line 964, in request
    self._send_request(method, url, body, headers)
  File "/opt/dionaea/lib/python3.2/http/client.py", line 1002, in _send_request
    self.endheaders(body)
  File "/opt/dionaea/lib/python3.2/http/client.py", line 960, in endheaders
    self._send_output(message_body)
  File "/opt/dionaea/lib/python3.2/http/client.py", line 805, in _send_output
    self.send(msg)
  File "/opt/dionaea/lib/python3.2/http/client.py", line 743, in send
    self.connect()
  File "/opt/dionaea/lib/python3.2/http/client.py", line 1105, in connect
    server_hostname=server_hostname)
  File "/opt/dionaea/lib/python3.2/ssl.py", line 168, in wrap_socket
    _context=self)
  File "/opt/dionaea/lib/python3.2/ssl.py", line 254, in __init__
    raise x
  File "/opt/dionaea/lib/python3.2/ssl.py", line 250, in __init__
    self.do_handshake()
  File "/opt/dionaea/lib/python3.2/ssl.py", line 429, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:390: error:14077458:SSL routines:SSL23_GET_SERVER_HELLO:reason(1112)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/dionaea/lib/python3.2/urllib/request.py", line 138, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/dionaea/lib/python3.2/urllib/request.py", line 366, in open
    response = self._open(req, data)
  File "/opt/dionaea/lib/python3.2/urllib/request.py", line 384, in _open
    '_open', req)
  File "/opt/dionaea/lib/python3.2/urllib/request.py", line 344, in _call_chain
    result = func(*args)
  File "/opt/dionaea/lib/python3.2/urllib/request.py", line 1156, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/opt/dionaea/lib/python3.2/urllib/request.py", line 1125, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 1] _ssl.c:390: error:14077458:SSL routines:SSL23_GET_SERVER_HELLO:reason(1112)>

Read more…

Kernel 2.6.38 introduced an API to access the kernel crypto API from userspace. While there was a port of BSD's cryptodev for linux which basically provides the same functionality, the cryptodev code never made it into the mainline of the kernel.

Accessing the kernels crypto API from userspace allows making use of crypto hardware, which can't be accessed from userspace directly. Hardware accelerated cryptography as provided by VIA Padlock¹⁾ and Intel AES-NI²⁾ can be accessed from userspace directly, so you do not need AF_ALG at all, but AMD Geode processors AES cryptography is - contrary to Padlock and AES-NI - not an instruction³⁾ and therefore can't be accessed from userspace.
You may be interested in the discussion of af_alg vs cryptodev performance ⁴⁾ and the raw numbers and benchmarking software too ⁵⁾.

As I own AMD Geode powered hardware (ALIX), I decided to play with AF_ALG.

Read more…

dionaea does https, at least tcp/443 is open and you can establish a tls connection. As you need certificates for ssl, and I felt it was easier to create a self signed certificate during startup than having to mess with openssl to create a self signed certificate, dionaea creates a self signed certificate for ssl services by default.

The EFF decided to grab all https ssl certificates ¹⁾, make a torrent and have you play with them.
When the torrent was available, the tracker was down, and for me having a look on the data offered was postponed.
Lately I remembered, downloaded the 4GB torrent, unpacked the file and ended up with ~20G csv.

/tmp/eff/csv-db-files$ ls -alh
total 20G
 13G ... all-certs.csv
250M ... all-names.csv
3.0G ... certs-seen.csv
3.6G ... valid-certs.csv
 74M ... valid-names.csv

The code dionaea uses to create the self signed certificate uses static strings:

X509_NAME_add_entry_by_txt(name,"C", MBSTRING_ASC, (const unsigned char *)"DE", -1, -1, 0);
X509_NAME_add_entry_by_txt(name,"CN", MBSTRING_ASC, (const unsigned char *)"Nepenthes Development Team", -1, -1, 0);
X509_NAME_add_entry_by_txt(name,"O", MBSTRING_ASC, (const unsigned char *)"dionaea.carnivore.it", -1, -1, 0);
X509_NAME_add_entry_by_txt(name,"OU", MBSTRING_ASC, (const unsigned char *)"anv", -1, -1, 0);

These static strings are part of the certificates:

openssl s_client -connect HOST:443 < /dev/null | openssl x509 -outform DER | openssl sha1
depth=0 /C=DE/CN=Nepenthes Development Team/O=dionaea.carnivore.it/OU=anv
verify error:num=18:self signed certificate
verify return:1
depth=0 /C=DE/CN=Nepenthes Development Team/O=dionaea.carnivore.it/OU=anv
verify return:1
DONE

Running grep on the CSV showed some dionaea deployments:

grep dionaea /tmp/eff/csv-db-files/all-certs.csv > dionaea.csv

I used python to retrieve the addresses from the csv:

f = open("dionaea.txt","wb")
c = csv.reader(open('dionaea.csv', 'rb'), delimiter=',', quotechar='"')
for i in c:
    f.write("%s\n" % (i[12],))
f.close()    

And to check if the hosts were still alive, I decided to use the software the EFF used to retrieve all the certificates:

/tmp/eff/ssl-observatory/scan$ ./FasterCertificateGrabber.py -f dionaea.txt
...
Got 5 complete and 0 partial certs out of 154

So, of those hosts only 5 were still alive, which is not that surprising as the majority of addresses seemed to be a single deployment in Australia.

While creating self signed certificates on the fly is convenient, it is easy to fingerprint and index.

At the moment there is no way to mitigate this kind of fingerprinting, it would required user generated certificates -or at least random or dictionary strings for the static part of the certificate- and some bits to load user generated certificates instead of generating them on the fly during startup.