I've had some problems with ssl lately, here is what I found to be the problem/solution.
The problem is pretty easy, inability to access https services, mwanalysis.org may serve as an example here.
I was able to reproduce the problem using python(3.2):
import urllib.request
url="https://mwanalysis.org"
a = urllib.request.urlopen(url)
Traceback (most recent call last):
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 1122, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "/opt/dionaea/lib/python3.2/http/client.py", line 964, in request
self._send_request(method, url, body, headers)
File "/opt/dionaea/lib/python3.2/http/client.py", line 1002, in _send_request
self.endheaders(body)
File "/opt/dionaea/lib/python3.2/http/client.py", line 960, in endheaders
self._send_output(message_body)
File "/opt/dionaea/lib/python3.2/http/client.py", line 805, in _send_output
self.send(msg)
File "/opt/dionaea/lib/python3.2/http/client.py", line 743, in send
self.connect()
File "/opt/dionaea/lib/python3.2/http/client.py", line 1105, in connect
server_hostname=server_hostname)
File "/opt/dionaea/lib/python3.2/ssl.py", line 168, in wrap_socket
_context=self)
File "/opt/dionaea/lib/python3.2/ssl.py", line 254, in __init__
raise x
File "/opt/dionaea/lib/python3.2/ssl.py", line 250, in __init__
self.do_handshake()
File "/opt/dionaea/lib/python3.2/ssl.py", line 429, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:390: error:14077458:SSL routines:SSL23_GET_SERVER_HELLO:reason(1112)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 138, in urlopen
return opener.open(url, data, timeout)
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 366, in open
response = self._open(req, data)
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 384, in _open
'_open', req)
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 344, in _call_chain
result = func(*args)
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 1156, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 1125, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 1] _ssl.c:390: error:14077458:SSL routines:SSL23_GET_SERVER_HELLO:reason(1112)>
Curl suffered the same problem:
curl -vvv "https://mwanalysis.org"
* About to connect() to mwanalysis.org port 443 (#0)
* Trying 131.188.31.200... connected
* Connected to mwanalysis.org (131.188.31.200) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* error:14077458:SSL routines:SSL23_GET_SERVER_HELLO:reason(1112)
* Closing connection #0
curl: (35) error:14077458:SSL routines:SSL23_GET_SERVER_HELLO:reason(1112)
For Python it is possible to provide a context to the urllib.request.HTTPSHandler and use this ssl context instead of the reasonable default:
# ...
class HTTPSConnection(HTTPConnection):
# ...
def __init__(self, host, port=None, key_file=None, cert_file=None,
strict=_strict_sentinel, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None, *, context=None, check_hostname=None):
# ...
if context is None:
# Some reasonable defaults
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
context.options |= ssl.OP_NO_SSLv2
# ...
http://hg.python.org/releasing/3.2.2/file/7fbcde1b5229/Lib/http/client.py#l1064
To use this with urllib:
import ssl
import urllib.request
url="https://mwanalysis.org"
urllib.request.install_opener(urllib.request.build_opener(urllib.request.HTTPSHandler(context=ssl.SSLContext(ssl.PROTOCOL_TLSv1))))
a = urllib.request.urlopen(url)
In case you use tor via privoxy:
import ssl
import urllib.request
url="https://mwanalysis.org"
urllib.request.install_opener(urllib.request.build_opener(urllib.request.ProxyHandler({"http":"http://127.0.0.1:8118"}),
urllib.request.build_opener(urlliburllib.request.HTTPSHandler(context=ssl.SSLContext(ssl.PROTOCOL_TLSv1))))
a = urllib.request.urlopen(url)
Passing -3 or -ssl3 to curl solves the problem:
curl -k -3 -vvv "https://mwanalysis.org" > /dev/null
* About to connect() to mwanalysis.org port 443 (#0)
* Trying 131.188.31.200... % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0connected
* Connected to mwanalysis.org (131.188.31.200) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Server hello (2):
{ [data not shown]
* SSLv3, TLS handshake, CERT (11):
{ [data not shown]
* SSLv3, TLS handshake, Server key exchange (12):
{ [data not shown]
* SSLv3, TLS handshake, Server finished (14):
{ [data not shown]
* SSLv3, TLS handshake, Client key exchange (16):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Finished (20):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
{ [data not shown]
* SSLv3, TLS handshake, Finished (20):
{ [data not shown]
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* subject: C=DE; ST=Baden-Wuerttemberg; L=Mannheim; O=Universitaet Mannheim; OU=Lehrstuhl fuer Praktische Informatik 1; CN=mwanalysis.org
* start date: 2010-06-23 09:38:09 GMT
* expire date: 2015-06-22 09:38:09 GMT
* common name: mwanalysis.org (matched)
* issuer: C=DE; ST=Baden-Wuerttemberg; L=Mannheim; O=Universitaet Mannheim; OU=Rechenzentrum; CN=RUM-CA-G Zertifizierungsinstanz; emailAddress=rum-ca@rz.uni-mannheim.de
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> GET / HTTP/1.1
> User-Agent: curl/7.21.3 (x86_64-pc-linux-gnu) libcurl/7.21.3 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
> Host: mwanalysis.org
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 07 Oct 2011 15:34:03 GMT
< Server: Apache/2.2.20 (Debian)
< X-Powered-By: PHP/5.3.8-1
< Set-Cookie: PHPSESSID=8f9ac9b7072322885fb95d0405ef1cee; path=/
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< Vary: Accept-Encoding
< Content-Length: 4511
< Content-Type: text/html
<
{ [data not shown]
100 4511 100 4511 0 0 21794 0 --:--:-- --:--:-- --:--:-- 31545* Connection #0 to host mwanalysis.org left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
} [data not shown]
Interestingly wget works fine for me
wget --no-check-certificate -O /dev/null https://mwanalysis.org
Resolving mwanalysis.org... 131.188.31.200
Connecting to mwanalysis.org|131.188.31.200|:443... connected.
WARNING: cannot verify mwanalysis.org's certificate, issued by `/C=DE/ST=Baden-Wuerttemberg/L=Mannheim/O=Universitaet Mannheim/OU=Rechenzentrum/CN=RUM-CA-G Zertifizierungsinstanz/emailAddress=rum-ca@rz.uni-mannheim.de':
Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 4511 (4.4K) [text/html]
Saving to: `/dev/null'
100%[===========================================================================>] 4,511 --.-K/s in 0s
even with SSLv23_client_method:
bool
ssl_init ()
{
SSL_METHOD *meth;
// ...
switch (opt.secure_protocol)
{
case secure_protocol_auto:
meth = SSLv23_client_method ();
break;
#ifndef OPENSSL_NO_SSL2
case secure_protocol_sslv2:
meth = SSLv2_client_method ();
break;
#endif
case secure_protocol_sslv3:
meth = SSLv3_client_method ();
break;
case secure_protocol_tlsv1:
meth = TLSv1_client_method ();
break;
default:
abort ();
}
ssl_ctx = SSL_CTX_new (meth);
Wireshark showed some differences when using curl with -tls1/-ssl3 or no option at all.
In case of no option at all, it would have openssl sent a TLSv1 Record Layer: Handshake Protocol: Client Hello with the Extension: "server_name", and the server would respond with an TLSv1 Record Layer: Alert (Level: Warning, Description: Unrecognized Name), causing the client to abort.
Using -tls1, curl would fail the same as it fails for no option.
Using -ssl3, curl would sent the SSLv3 Record Layer: Handshake Protocol: Client Hello without the server_name extension.
Python with the defined opener for TLSv1 will sent the server_name extension, receive the TLSv1 Record Layer: Alert (Level: Warning, Description: Unrecognized Name) but ignore it somewhere.
wget does not set the server_name extension, does not receive an Alert and works fine by default therefore.
The OpenSSL version used on my side is “OpenSSL 0.9.8o 01 Jun 2010”.
Thanks for this! I was having the same problem with a PHP script using the PHP Curl objects. For me it started when I was running the script on an Ubuntu Natty box trying to talk to an Ubuntu Oneiric server. Natty to Natty worked fine, Oneiric to Oneiric worked fine. Oneiric to Natty worked fine. It must be something in the specific OpenSSL implementation in Natty for me. For reference, I had openssl version 0.9.8o-5ubuntu1 on the machine failing to make a connection, so perhaps it's the 0.9.8o client version specifically.
Another interesting tidbit, if I proxied the SSL connection through Pound on the server, the problem went away.