Cooperative Oterm, part I: Sharing the ajax xterm.

Oterm, onion's web ajax-powered xterm emulator is getting cooperative mode. With it you could send a link to another user and connect straight to a Linux terminal. It will perform the sharing, chat and even NAT transversal where possible.

In these posts I will explain the steps that were performed to get to the cooperative Oterm.

Source code as it develops can be found at github.

The several readers one writer problem.

Shared terminal example

In order for several readers to be able to use the same terminal there has to be some mechanisms to allow to have several readers of the output of the executing program in the terminal.

Basically we have one buffer with the latest data from the terminal, and then several web readers each which can be on different position on the stream. It is important to note that each terminal normally joins in different points of the stream, so we must send the buffer contents so they can show something to the user, and then wait for new data as it appears, with possible delays meaning that one terminal gets data from one point and the others from another.

Internally Oterm does not use (yet?) websockets, just connects to one URL, the out stream, and writes posts to another, the in stream. The out stream is blocking, which means that the browser will ask for the URL but will not get the answer until some data is ready. This is done using one thread per connection.

Write to the internal buffer
So the first change needed was to change the behaviour of blocking on the client side until some data was ready, and make the spawned command (normally bash) to write when data is ready to an internal buffer. This is a circular buffer on which we just write at the end overwritting old data. The default size is 16KB (a terminal size of  120x136 full of data, for example).

To do this we used the new poller facilities, which make use of a thread of events which get woken up when new data is in a read stream. Perfect for this. Now when data is ready we signal using pthread conditions the waiting thread on the connection to write the available data.

So now we don't have a connection waiting for the data from the terminal, but for just a pthread condition.

Clients on different stream positions
Once we have the data in the buffer, when the client is waiting for data we must know from where it needs the data. We can not rely on having a server-side marker for each connection given the connection-less nature of HTTP, and doing tricks here might not be wise as we might lose some characters if some request is not properly sent. Also we don't know when a connection is not used any more, and using timers is not nice given that we block indefinitely.

So as solution of all this, we send at the end of every packet the current position on the stream. So now when the user ask for data it asks for data from a specific position. If there is data, the server returns it promptly. If there is not, then we wait on the condition for new data ready. For this we use a new control command specific for Oterm. Users nor applications should be aware of this as the emmiter is the server and the client intercepts it and interprets it.

We moved the session problem about from where on the buffer to ask to the client, where we have proper state tracking.

Future enhancements
On future articles I will talk about opening NAT for onion processes,  so user from outside of our network can access our Oterm. This is a security sensitive issue, as user may open its computer straight to the World Wide Web, but UPnP gives us tools, and in some situations its the perfect way to cooperatively work on a Linux terminal.