YATP: A Torrent Streaming Plugin for Kodi

 May 23, 2016  


My YATP (Yet Another Torrent Player) plugin for Kodi mediacenter is not a new project. I stated it a wile ago partly out of curiosity, partly because of practical reasons because I needed a torrent player that satisfied my needs. The solutions that existed at that time did not satisfy me. AceStream was (and stil is) proprietary and did not support all platforms that run Kodi, Pulsar (now reincarnated as Quasar) deleted videofiles immediately after playback, facilitating torrent "hit'n'running" and Torrenter developed by some Russian guy was too messy from my point of view (no offense meant).

Because implementing the BitTorrent protocol from the scratch was way over my head, I choose libtorrent library that has Python binding, meaning that it can be used as a regular Python module. Naturally, this binary module has to be compiled for your target platform, but fortunately guys from the unofficial Russian Kodi forum managed to compile libtorrent modules virtually for every platform that can run Kodi.

Problems

The libtorrent library has its own features that facilitate torrent streaming, like set_sequential_download method and setting piece deadlines. However, according to the libtorrent documentation, they do not guarantee that necessary pieces will be delivered in time when they are needed for playback. And this is a problem because Kodi badly tolerates broken audio/video data. Best case scenario: you'll get visual or sound artifacts, worst case: the playback will be interrupted. So I needed a method that would guarantee that all necessary pieces will be delivered in time.

Another problem closely connected with the previous one is that Kodi tries to cache a file being played as much as possible, meaning that it can still swallow into its cache the parts of the video that haven't been downloaded yet (means random garbage). So I needed a method that would control serving a video to Kodi in order to prevent caching parts that aren't downloaded.

And the last problem that I wanted to solve is jumping/seeking to arbitrary parts of a video, allowing a user to skip parts that they do not want to watch. This means that I need somehow to reset the sliding window to a new position and start downloading torrent from a new point.

Solutions

After studying some theory, including this work, I selected the fixed size sliding window algorithm as the most simplest to solve the 1st problem. Naturally, this requires torrent download speed to be greater than the total bitrate of a videofile plus protocol overhead. Also it is worth to note, that, technically, sequential downloading a file is not actually "streaming". We just cleverly trick a video player, Kodi in our case, to play an incomplete videofile while missing pieces are being downloaded in the background.

The fixed size sliding window algorithm can be illustrated by the following figure:

With this algorithm torrent pieces are downloaded within some set or a "window" of fixed length, and when a piece at the beginning of the window is downloaded the window moves (slides) forward by one pieces until it reaches the end of the videofile.

To solve the 2nd and 3rd problems I decided to serve videofiles to Kodi from a local webserver. First, it allows to control file serving itself and also to get the information to what part of a video Kodi wants to jump/seek. The latter is possible because Kodi issues a GET request with Range header when streaming from HTTP sources to request the necessary part of a video, so its easy to calculate which piece of the torrent Kodi wants now and reset the sliding window to that piece.

For the 2nd problem I implemented the algorithm that I called "controlled file serving". It is based on the fact that a WSGI server can serve response content from any iterable object provided by a Python WSGI application. In my case this iterable is a Python generator function that yields torrent pieces to Kodi. This function checks each torrent piece if it is actually downloaded before serving it to Kodi. If a piece is not yet downloaded, the function prioritizes it and pauses playback if necessary. Naturally, this means that with low download speed playback will be paused too often to wait for missing pieces, but I decided that it is a "lesser evil" than video/audio artifacts or even interrupted playback.

Conclusion

My solution proved to be efficient for streaming torrents with small to medium pieces, that is, from hundreds of kilobytes to 1 megabyte in size. But larger torrents that have bigger pieces varying from several megabytes to more than 10 megabytes, proved to be difficult to stream. This means longer buffering times, frequent pauses to wait for missing pieces, inability to jump to a necessary part of a video. This is not surprising because with pieces measured in megabytes you can hardly call this process "streaming" but rather "step by step downloading." I think that the ability to control torrent download on sub-piece level could help here but, unfortunately, libtorrent does not provide such facilities. The smallest part that you can control programmatically is a torrent piece.

You can find my torrent streaming solution implemented as a video plugin for Kodi in my GitHub repository. I decided to keep my plugin legally "clean", meaning that it does not work with any specific torrent sites but rather implements generic torrent streaming functionality. However, a rich API allows to use YATP as a torrent player for other Kodi plugins.

  BitTorrentKodiPluginPython