As distributed sensors are desireable, I had another look on xmpp over the weekend. XMPP is basically an xml stream for instant messaging, which can be used for other things too.

The good news first, it works:

screenshot from the reporting using psi, I had to change the rendering to escape < and >, else psi would not render it …

I've had some xmpp reporting code for dionaea already, it made use of libloudmouth, was written in c and basically worked.
But I thought this was easier to do in python, so I decided to rewrite the logxmpp using python …
I found two note able xmpp frameworks for python, xmpppy and SleekXMPP. For both frameworks, I did not like the API, they did not fit into the process-model dionaea uses. xmpppy is written for python2, and I expected problems when porting it to do python3 bytes vs python2 strings, SleekXMPP supports python3(.1), but it uses blocking pythons xml processing to receive streams. So, as dionaea offers tls connections by default for the python binding, I decided to write the required parts of the protocol myself, given the fact python3 has a large collection of xml processing facilities.

The python api offers xml.sax.xmlreader.IncrementalParser, which can be feed with new data once available - at least according to the documentation.

Unfortunately feeding is not implemented:

class IncrementalParser(XMLReader):
    """This interface adds three extra methods to the XMLReader ..."""
 
    def feed(self, data):
        """This method gives the raw XML data in the data parameter to
        the parser and makes it parse the data, emitting the
        corresponding events. It is allowed for XML constructs to be
        split across several calls to feed.
 
        feed may raise SAXException."""
        raise NotImplementedError("This method must be implemented!")

I've tried XMLParser fed using BytesIO, nothing worked.

So, pythons default Incremental xml parser is not useable, looking for alternatives I stumbled upon lxml, which provides an API similar to pythons own, but implements the features advertised. One has to install lxml from source, but I thought one more easy to install dependency does not really matter for dionaea.

Additionally, lxml provides xpath matching, which comes handy when retrieving messages from the xml stream.

So, I installed ejabberd from debian apt locally - after I failed installing from source to a dedicated directory (/opt/ejabberd), due to the exmpp library dependency, installed psi as client, and created some account (user@example.com) on the ejabberd and made sure psi can log in.
psi has a great feature, it can write all data sent and received to a console.

Afterwards I finally started writing the code required to login to an xmpp service.

Even though XMPP has great documentation, I decided to take a shortcut, looking at psi xml console, creating the xml objects in python, tostring them, and send the data.

As ejabberd advertised PLAIN authentication, the xmpp connection is meant to be tls encrypted anyway, and I was lazy, I decided to go for PLAIN authentication. As ejabberd does not support PLAIN auth, even though it is advertised during the handshake, this did not work at all. I spent some cycles on trying to figure out why … until some google research on the topic made me realize I had to go for sasl DIGEST-MD5 auth instead. Slightly frustrated … I had a look on the sasl docs, and decided to use the DIGEST-MD5 authentication code from xmpppy, adjust it to work with python3, and be done with it. After some time, dionaea was able to login to the xmpp service and receive messages from my psi xmpp client, next was joining a channel, and sending incident messages to the channel. psi did not render the xml messages, but the xml console showed, them, therefore I escaped the xml message for the above screenshot.

After basics worked, I was surprised to see messages sent by a client get replied to the client by the server, and I'll have to disable sending messages from sensors to other sensors as well.

My first server software was ejabberd. ejabberd seems to be the big player for xmpp services, it is pretty famous for massive scalability (500.000 clients on a cluster), but it is written in erlang, which is not a language I speak fluently. I even found the code I'd have to modify to disable message routing between sensors. The plan is to give sensors a special channel affiliation, vistor or participant, and not relay messages from this affilitation to other channel occupants with the same affilitation.

process_groupchat_message(From, #xmlel{name = 'message'} = Packet,
			  StateData) ->
    Lang = exmpp_stanza:get_lang(Packet),
    case is_user_online(From, StateData) orelse
	is_user_allowed_message_nonparticipant(From, StateData) of
	true ->
	    {FromNick, Role} = get_participant_data(From, StateData),
	    if
		(Role == moderator) or (Role == participant) 
		or ((StateData#state.config)#config.moderated == false) ->
		    {NewStateData1, IsAllowed} =
			case check_subject(Packet) of
			    false ->
				{StateData, true};
			    Subject ->
...
			case IsAllowed of
			true ->
				lists:foreach(
					fun({_LJID, Info}) ->
						ejabberd_router:route(
						jid_replace_resource(
						StateData#state.jid,
						FromNick),
						Info#user.jid,
					Packet)
					end,
					?DICT:to_list(StateData#state.users)),
				NewStateData2 =
				add_message_to_history(FromNick,
						       From,
						       Packet,
						       NewStateData1),
			    {next_state, normal_state, NewStateData2};

ejabberd's src/mod_muc
Besides the codes indenting being pretty bad (mixed tabs and spaces) and from beeing totally unintuitive, I was unable to figure out how to filter the list of user which will receive a message sent to a channel, based on the channel affiliation.

?DICT:to_list(StateData#state.users)),

I got over it, when I found prosody, which is written in lua, got much easier to read and modify code, and debian packages for all required dependencies.

I easily found the code to set the default affiliation for a channel user:

function room_mt:get_default_role(affiliation)
	if affiliation == "owner" or affiliation == "admin" then
		return "moderator";
	elseif affiliation == "member" or not affiliation then
		return "participant";
	end
end

the code to get a channel users affiliation:

function room_mt:get_affiliation(jid)
	local node, host, resource = jid_split(jid);
	local bare = node and node.."@"..host or host;
	local result = self._affiliations[bare]; -- Affiliations are granted, revoked, and maintained based on the user's bare JID.
	if not result and self._affiliations[host] == "outcast" then result = "outcast"; end -- host banned
	return result;
end

plugins/muc/muc.lib.lua

and the code where message broadcasting to the channel is done:

function room_mt:handle_to_room(origin, stanza) -- presence changes and groupchat messages, along with disco/etc
	local type = stanza.attr.type;
	local xmlns = stanza.tags[1] and stanza.tags[1].attr.xmlns;
	if stanza.name == "iq" then
...
	elseif stanza.name == "message" and type == "groupchat" then
		local from, to = stanza.attr.from, stanza.attr.to;
		local room = jid_bare(to);
		local current_nick = self._jid_nick[from];
		local occupant = self._occupants[current_nick];
		if not occupant then -- not in room
			origin.send(st.error_reply(stanza, "cancel", "not-acceptable"));
		elseif occupant.role == "visitor" then
			origin.send(st.error_reply(stanza, "cancel", "forbidden"));
		else
			local from = stanza.attr.from;
			stanza.attr.from = current_nick;
			local subject = getText(stanza, {"subject"});
			if subject then
...
			else
				self:broadcast_message(stanza, true);
			end
			stanza.attr.from = from;
		end

plugins/muc/muc.lib.lua

So, I figured it will be trivial to set the default role to visitor, allow messages from visitors, but route them to non-visitor occupants only …

Even though the client-code already works, it … needs some touches, and additionally I want to be able to set different locations/channels for different types of events, so there can be a dedicated channel for files which get streamed to the server.
For the server-code, the mentioned hacks to allow visitors to send messages to non-visitors in a channel have to be done.
For the backend, there is no code yet, I'll try to provide some basic xmpppy-bot which stores files, which get streamed to a channel by the sensors, to harddisk.
Afterwards …, I got to commit to svn.

Finally, volunteers to test it.