The automatic downloading system

Ever wanted to go home, start up your XBMC and see that episode of the TV-show you wanted to watch yesterday, but forgot to watch on TV. Or just wanted that every TV-show that you follow just appears in your XBMC as soon as it is available on the internet? Look no further, as this post will guide you through setting up a system that does exactly that. Please note that it took some time for me to set this system up, which means that this setup will take some time. I would recommend sticking as close to this setup as possible unless you’re certain that it would not make a difference.

Please note, I prefer and recommend YaST to change the various configurations and stuff but I include the bare commands so people who can’t/won’t use YaST can also use this system.

Requirements

Hardware

This is quite simple, a server running linux. This is written to be installed on a system running OpenSUSE 12.3, but if you know how to translate the specific commands, you can also use debian, fedora and the like. You could also use a Mac. Unfortunately Windows is not supported. But hey, linux is free… and so is VirtualBox. (And you should not run windows as a server, asking for problems)

Software

Well I go very specific on this, you can try to use different ingredients but you are on your own. I know that this setup works as I use it myself.

• Transmission-daemon: This will download all your ‘legit’ TV-show downloads</li>
• Unrar: unrar some torrent files.</li>
• flexget: This will look for new downloads</li>
• Some custom written, poorly documented scripts: These will mainly do various stuff like unrar and moving tv-shows to your xbmc folders</li>

Other

Well, only some understanding of the basic linux system and scripting. And how to write configuration files. Perl is also handy to know, but not strictly necessary.

Preparation

In order to begin, you should install the various packages:

zypper in unrar

The transmission-daemon package can be found in the repository Gnome:Apps on the OpenSUSE build service:

zypper ar http://download.opensuse.org/repositories/GNOME:/Apps/openSUSE_12.3/ Gnome_Apps
zypper in transmission transmission-daemon

Unfortunately, the flexget package isn’t packaged by default. But the website has a very nice step-by-step process of guiding you through it. www.flexget.com

And for those who want it written out: Get python:

zypper in python python-xml python-pip

Install flexget:

pip install flexget

And you should get my custom scripts:

cd /usr/local/bin
wget https://danielsoncksolutions.nl/scripts/torrent-complete
wget https://danielsoncksolutions.nl/scripts/series-complete
wget https://danielsoncksolutions.nl/scripts/movecompletedtorrents
chmod ugo+rx torrent-complete series-complete

You can also install them in /usr/bin, but as a linux tradition, it’s better to use /usr/local/bin, as it will keep the default folders clean.

Setup

The following pieces are taken from my system and they have been there for almost a year. That means that although I tried to translate the files to a generic form, there may be errors inside them.

Users

As it increases security and it simplifies some configuration, it’s recommended to make a dedicated user for transmission. Although some people would use a system user for this, I rather not as it gives the ability to use ssh and ftp with your transmission account. Which is very useful.

useradd transmission
passwd transmission # not strictly necessary but useful for ftp/ssh

Destinations

You’ll have to determine where you want your TV-Shows to end up. I’ll assume that the files will go to /home/user/Videos/TV-Shows.

Transmission

In order to have some automatic systems, there are some tweaks necessary.

You’ll probably want to use a custom settings.json file:

cd /var/lib/transmission/.config/transmission/
wget http://dss.nl.eu.org/scripts/etc/transmission/settings.json

You should change the password and possibly some other settings. Also note that you should only edit this file while transmission isn’t running, otherwise it will overwrite the settings.

At this moment, you have an automatic transmission daemon which can run the scripts. Although this system would probably work, it won’t search for new TV-Shows. And you’ll probably won’t understand how the two automated scripts work and can be customized.

Automatic scripts

In order to use the two provided scripts, I’ll try to explain how they work and how they can be tweaked.

The torrent-complete script is very basic and it will just extract .rar files (and remove them after successful extraction)  and call the series-complete script. You do need the unrar program.

The series-complete script is very complex. It does a couple of things:

• Hardlink the files to a specific TV-Show folder using the format: <tv-show>/Season #/<s>x<e> - <tv-show>.mkv

  Like: Stargate SG-1/Season 5/5x10 - Stargate SG-1.mkv

There are some things that can be tweaked:

• The path where new TV Shows can be found
• The path where you want the TV Shows to end up
• Names of releasegroups that should be filtered out of the names

Formats understood by this powerful script:

S## E## ##x## Season # Ep# Ep. # Episode #

More indications may be given in the same filename:

S01 E01 E02
01x01 01x02

If no proper data is found, it will try to find a SEE or EE value. That is: one digit for season and two for episodes or only two digits for episodes. This is dangerous as x264 would be seen as 2x64 and the title Warehouse 13 would be seen as episode 13. Therefore it’s a last resort. If no season information is used it falls back to first season.

And at last the movecompletedtorrents. This script will remove the completed torrents from transmission because transmission gets full really fast if you’re using flexget.

You should include a line in your crontab:

su transmission
crontab -e
@hourly /usr/local/bin/movecompletedtorrents

Flexget

At last, to fully automate the process, you can configure flexget. Their website gives you all the information you need. The only extra thing you should remember is that you must set the path, and you should choose a path with Downloads and Series in the full path. Like the path mentioned earlier.

Since they included a useful templating system, I have a basic settings file here:

templates:
  tv:
    regexp:
      reject:
        - swesub
      reject_excluding:
        - DIMENSION|IMMERSE|EVOLVE|COMPULSiON|ORENJI|ASAP|CTU|SiNNERS|HorribleSubs|2HD|DiABLO
    series:
      settings:
        normal:
          set:
            path: "/home/transmission/Downloads/Complete/Series//"
    transmission:
      host: localhost
      username: transmission
      password: secret
      port: 9091

tasks:
  tvshow:
    rss: http://kickass.to/usearch/TV+Show/?rss=1
    template: tv
    series:
      normal:
      - TV Show
...

You basically give the task a unique name, enter the rss feed location, choose the tv preset and select the series you want (here it’s called ‘TV Show’). Thanks to the preset system, you only have to enter 6 lines for every TV Show. It will correctly set the path, exclude quality you don’t want and send it to transmission for download.

And this is the complete guide to get a fully automatic system which will download TV-Shows for you and format the filenames to a compatible format for XBMC. You would probably want to install the automatic library update plugin.

(If you love anime as much as I do, here is a post for you: Automatic anime download system and recent improvements to this system found here: Automatic anime and tv show download system improvement