NoVNC with audioOctober 26 2020 23:32:43

Portability is very useful - and I like being able to take my workspace everywhere. For a while now I've had a VPS with the german provider Contabo - and so far I've only used it for shell access. Of course I'm running Gentoo so transforming the server to a fully fledged virtual desktop isn't too difficult - just emerge all the relevant packages and start tigervnc.

Of course if everything was this easy - I wouldn't be writing a blog post about it. While tigervnc has sensible defaults on Gentoo - and getting a working VNC connection is fairly easy next comes the tough part - access. For this I've installed noVNC. When installed on a webserver this VNC client allows you to access your VNC server through a webbrowser. This VNC server however has some shortcomings - like all VNC servers it does not support audio streaming. A bit of googling reveals that people are already experimenting with audio support.

Of course things aren't as straight-forward as they seem - the code in this pull request has a few flaws. The first one being that I mainly work on a Macbook nowadays - and they do not support webm audio. To enable audio support that works on any computer I've used the older mp4 audio stream format. Furthermore I've written a small program in c that pipes the output from a pulse sink to a tcp socket. This is then piped through websockify and streamed to the client. With this I've achieved low-latency audio support for any modern browser :) Here is the end result, gentoo in my browser - indistinguishable from the real thing.


In order to achieve this look I've modified the files for NoVNC-lite, added automatic reconnects and a small menu bar for display quality settings - and added the files "vnc.js" and "webaudio.js". Webaudio is the implementation of the audio stream - and vnc.js contains the methods neccesary to invoke it. When you press your first key in the remote desktop environment - the webpage catches the event and starts the audio stream. This because I had to work around the fact that audio streams cannot be started automatically without user input in current browsers.
That's it - I've enjoyed writing this code - and I hope it's useful for others too. It's free software, so use it as you like.

Update ( 31 oct 2020 ): I've updated websockify to support current SSL methods and fixed copy-paste for VNC servers without clipboard. See novnc_current.zip below.