Hacking together a simple chromecast alternative in under 3 hours using GStreamer
Hey guys!
For lack of a blog; this is a little story of how I got so annoyed from a small inconvenience, that I used it as an excuse to play with Rust and GStreamer again (god I love GStreamer, and the fact that it has awesome Rust bindings).
Connected to my TV, I have a raspberry with Kodi/XBMC running on it. Since quite some time I'm annoyed by the fact that there's no way to somehow easily (couple of clicks max.) stream any kind of media to Kodi, to view it on the big screen, like Chromecast does. Occasionally for a long time now, I've been polling the Kodi Forums for ways to achieve this Chromecast functionality, but all I ever found were multiple pages of Admins, Developers and users fighting each other over whether the current functionality of Kodi is enough for every use-case, or whether it's not and whether someone was in the progress of adding Chromecast Receiver support to Kodi etc. Frustratingly, searches via our grand overlord Google always seemed to prefer telling me about Kodi's functionality to Chromecast itself (so, the opposite direction), whatever that's supposed to be good for.
Somewhere on page 5 (I don't know the exact number, could just as well have been 2) of a heated .. let's call it "discussion" .. a developer said that it's not possible to add Chromecast Receiver support to Kodi because while the Chromecast SDK is easily available, the receiving part is strictly proprietary and very hard to pick apart. Because apparently, both Chromecast sender and receiver connect to some google service, even though the communication between both happens locally. At that point, I was already in rage mode and started having flashbacks of a couple years earlier, when I found out that there will also never be an open-source Android Auto implementation (the sending part on the smartphone), for very similar reasons.
Further down in the thread, a developer said that Kodi already supports Youtube (you can send a Youtube link to Kodi and it opens it with its locally built-in Youtube player) as well as "UPNP/DLNA", and that that would be enough. Now, I want you to know that at that moment in time, the only thing I knew of under the "UPNP/DLNA" keyword was a standardized protocol for media servers. So almost a protocol akin to SMB/CIFS and NFS: The media server publishes a folder tree of playable files, that a client can browse through. And when a client selects an item, the server streams the file to the client for display. I had played with that very many moons ago, and let's say I didn't have a lot of fond memories from it. For today's streaming services, that would obviously not cut it, since they don't just simply stream a single video file to the browser for display. They use streaming protocols like WebRTC or HLS that involves signalling - on the one hand, because they need support for seamlessly varying quality levels while the video plays, and on the other hand, because then piracy is no longer just a visit to the developer panel - that'd be too easy. \sad ARRR noises\
Now, since that developer's statement seemed so obviously wrong with my knowledge at that time - especially since Kodi has no legal support for Netflix, Amazon Prime, and other video services like they do for Youtube - my blood preasure rose significantly.
Further down in the discussion, some developer added that, while Apple's Airplay (similar protocol to Chromecast) was currently implemented, the implementation was basically "on lifesupport", often times already falling apart. That is because, while the initial specification/design of the protocol was apparently very simple and elegant, its design then heavily escalated over time, with special ... "urls" ... and special handling for some content providers. Now at that point I thought: Waaaaait a minute!. "URLs"? What does he mean by "urls for certain providers"?? I was under the impression that AirPlay and Chromecast were using the browser to download the video data (of Netflix, Youtube, or whatever) like they normally would during playback, and simply forwarding that data to the player. Seemed like the most obvious choice with the least effort to me. When I thought a little about what the alternative was, I came to the conclusion that the sender part must be constructing URLs for the content, then send that URL to the destination and then that destination has to do all the heavy lifting of downloading the video data itself.
After that "shock" had settled, I thought that UPNP/DLNA couldn't be worse anyway, so let's have a look at what was possible with Kodi. Some user mentioned the App BubbleUPnP, that - according to its description - allowed you to stream media files on your phone to the TV (at first, very much confirming my misconception with UPnP/DLNA mentioned above). I tried the app anyway. It basically works like this: You select a file on your smartphone, then you have to select a "player" for the file. In that list of players, my Kodi instance appeared. That already seemed weird to me, I thought it worked the other way around. I selected it, pressed play - and instantly the video appeared on my TV.
Wait. What? At that point I read into UPnP/DLNA to understand what magical tale had just unfolded in my hands. Turns out that protocol also works in the opposite direction. Kodi advertises a service that basically allows you to remote control it via some basic actions. I further looked into it and saw that the "play this now, slave!" command just sent an URL to the renderer, which the renderer then loads and displays. Sounds familiar, where have I heard that already..?
That gave me an idea! A couple years earlier during Corona, together with a friend, I built a small remotedesktop application. Stunningly enough, this "application" started off as 2 bash scripts containing 6 lines of GStreamer commandline each. (No really, I love GStreamer!)
Of course, the scripts didn't support any kind of NAT hole punching or any other kind of p2p network connection technique that would be needed to make a working solution out of it, it only supported the local network - which we forced through a poor VPN.
The two scripts basically set up a one-directional rtp:// stream, which is a real-time streaming protocol that takes over all kinds of tasks, such as transporting the actual data and synchronizing clocks for audio/video pakets between both computers, extended by an rtcp:// connection which is the signaling part that manages stuff like requesting a retransmission of a lost (udp) data packet etc. Kodi already needs to support some kind of network transport protocol for the data streamed from said Android app ... so how far off was the idea that it may support rtsp, rtp and rtcp...?
That piqued my interest! I searched for a crate that supports the media remote control command via UPNP/DLNA that I learned about earlier in order to force Kodi to open an arbitrary URL. I quickly found upnp-client, which funnily enough, had exactly what I needed as code example on the front page. So I opened a new project, copy pasted the code, hit compile ... and was annoyed about myself for having forgotten to add the crate as dependency first. When I ran it, I was pleased to see a big white bunny walking across my TV - so I was done with the remote controling part in about... 30 seconds. Nice!
Now I needed an RTSP stream. After attempting to play 10 URLs all across the internet (first using the media player on my desktop) which all turned out to be dead, I luckily remembered that VLC didn't used to be a media player back in the early days. I started VLC, gave him a video file and instructed VLC to provide an rtsp stream from it. I pasted that URL in the remote control code, hit F5 - and wondered why my TV was unable to open rtsp://localhost:8554. It worked flawlessly when I opened the stream for test reasons on my PC after all..! After I corrected the url to point to my computer, the TV lit up! Well, it was mostly green and full of decoding errors, but sometimes I was sure I saw something that could've resembled a bunny... or bigfoot's envious brother, for that matter. The stuttering sound was a good match for the framerate of the slideshow that I was privileged to witness.
Seeing that TV lit up with an image and that nightmarish sound (never mind the decoding errors) made my adrenaline levels shoot up. Kodi natively supports playing rtsp:// streams, awesome!
... What do I do with that now?
After very quickly dismissing the idea of hacking this into the Firefox codebase (I'm a little dumb, not insane), I came to the conclusion that the most versatile solution was to simply capture my screen, like we did with the remotedesktop experiment, capture the audio output of the running applications and send both alongside each other to Kodi. The rtp:// protocol handling the task of transmitting the timestamped data ensures that both video and audio couldn't shouldn't drift apart.
Since GStreamer has an easy-to-use rtsp server implementation, I added it to my small project and hacked the basic rtsp server example into the already existing remote control code.
After changing the launch pipeline to produce some h264-encoded test-data using (audiotestsrc and videotestsrc), I hit F5 and was pleased to hear an incredibly annoying beep sound from the other room 3 seconds later. (Have I btw. said that I love GStreamer already? No? I do, and some friends are already annoyed with me for bringing it up all the time!)
From then, getting everything working the way I wanted was just changing the launch commandline to do a screen and audio capture, live-encode both to h264 and vorbis respectively, and hitting F5 again. In total, that amounted to 92 lines of code - which I lost because I'm an idiot. Turns out that the temporary folder doesn't survive reboots (sample #91235 of this happening). Huh. Interesting.
If someone's interested: I extended it with support for both Wayland and X11, did some performance tuning and pushed the current state to https://github.com/seijikun/desktopcast. The GStreamer pipeline is constructed here.
It's still not a whole lot of code and might be interesting for some of you.
TL;DR
Tiny tool that streams your desktop and audio to Kodi using GStreamer, UPNP/DLNA, and rt(s/c)p://