Adding Torrents via my.pogoplug.com

If you have setup Transmission on your Pogoplug using my previous post, then this post will allow you to remotely (via my.pogoplug.com) add new torrent files and start them downloading to your home server.

Why?

I’ve been looking for a secure way to add Torrents to my Transmission server running on my Pogoplug, from a remote location.  This would allow me to start new torrents (during my lunchbreak at work for example), and have them finished and ready for me by the time I got home.  This is handy for starting larger files, such as new Ubuntu releases (and recently, the Microsoft Windows 8 Consumer Preview), without having to be in your local network,

Initially, I put the Transmission web interface on a public IP, but I’m not particularly comfortable with opening extra ports from my LAN to the internet, so I soon switched this off.  This morning I hacked together a solution that utilises the security that already exists with my.pogoplug.com (SSL, username/password authentication), and allows for adding a torrent file.

How

It works as follows:

  • You upload your selected .torrent file to a folder via my.pogoplug.com
  • A scheduled task (cronjob) checks the folder for new torrent files at a set interval (e.g. every 5 minutes)
  • If the scheduled task finds any new torrents, it adds them to the local Transmission server, then removes the .torrent file from disk
  • When you get home, your torrent is (hopefully) finished and ready for you.  Simple :)

Instructions

  • Log in to my.pogoplug.com
  • Select the drive that you would like to use for temporarily storing .torrent files, and create a folder names “Torrents” in the root of the drive:
  • SSH onto your pogoplug and perform the following:
# create the required crontab directory on our flash drive
mkdir -p /opt/var/spool/cron/crontabs
 
# link from the main drive to our flash
ln -s /opt/var/spool /var/spool
 
# create the transmission scheduler script
touch /opt/bin/transmission-scheduler.sh
vi /opt/bin/transmission-scheduler.sh

The next bit involves a little editing before we can copy and paste as the contents of the transmission-scheduler.sh script.  There are two variables at the top of this script that you will need to customise before this script will work – “pathToTorrentFolder” and “transmissionUrl”.

pathToTorrentFolder should be set to the filesystem path of your Torrent folder.  Use the output of “df -h” to find yours – my drive is mounted to “/tmp/.cemnt/mnt_sda1/” so my Torrents folder is “/tmp/.cemnt/mnt_sda1/Torrents/”.

transmissionUrl should be set to your Transmission web interface url.  For example, mine is “http://192.168.0.10:9091/transmission”.  It is really important that you follow the same format (“/transmission” at the end).  And make sure to have no trailing slash.

Now copy and paste the following script into the terminal (with YOUR custom variable values instead of mine :) ):

#!/bin/sh
 
pathToTorrentFolder='/tmp/.cemnt/mnt_sda1/Torrents/'
transmissionUrl='http://192.168.0.10:9091/transmission'
 
for f in $pathToTorrentFolder*.torrent ;
do /opt/bin/transmission-remote $transmissionUrl -a "$f"; rm "$f";
done

Save the file and exit Vi.  Now continue executing the following in the terminal:

# give execute permission on the scheduler script
chmod +x /opt/bin/transmission-scheduler.sh
 
# add a crontab entry to run the script at a specified interval
crontab -e

Paste the following in, substituting “5″ with your desired interval check (in minutes).

*/5 * * * * /opt/bin/transmission-scheduler.sh

Save the file and exit Vi.

Now continue in the terminal with the following:

# start the cron daemon
busybox crond

That should be it!

To test, grab a torrent file (may I suggest the Wired Magazine CC CD - http://www.clearbits.net/torrents/63-wired-magazine-creative-commons-cd), download it to your computer, then upload via my.pogoplug.com to your Torrents folder.

Go make a cup of tea, and by the time you come back, check the Transmission web interface.  If everything has gone to plan, Transmission is downloading the torrent and the .torrent file should have been deleted from the Torrents folder.

Any questions, give me a shout in the comments section.

Aaron

43 Comments

  1. Andy Bailey

    Hi Aaron,

    Nice article, however it is possible to simply configure Transmission to do all the hard work for you, rather than mess with cron jobs.

    If you edit the settings.json file and set watch-dir-enabled to “true” and configure the watch-dir parameter to point to a valid directlry, then you can simply drop a .torrent file into that directory. Transmission with then pick up the .torrent file (renaming it to be a .torrent.added file) and start the download. I find this useful as I can upload .torrent files directly from the pogoplug applicaiton on my mobile phone.

    HTH

    Andy

    • Aaron (Author)

      Hi Andy,

      Great tip, and definitely easier to setup than my technique :)

      Still, I had geeky fun putting this together, and I do like having the torrent files removed automatically when they have been started.

      Thanks for sharing.

      Aaron

      • Andy Bailey

        One thing I have noticed with Transmission however is the fact that it may well have a slow memory leak. I tend to find that after a couple of weeks, the drop-file method doesn’t work. Running a “top” shows that almost all the free memory has been used up. By killing off the transmission daemon and restarting it, everything works fine again.

        I was thinking of scheduling some kind of job to kill off and restart Transmission at regular intervals (say once per week). Your article may come in useful for configuring that. As I may have mentioned before, I’m by no means a Linux expery, but can muddle my way through the kind of instructions you post, so I’m sure this article will be a great help.

        Andy Bailey

        • Aaron (Author)

          Interesting to hear Andy; I haven’t experienced this issue myself. Let me know if you find the root of the problem (and maybe a solution!).

          To setup cron, you can probably just do the following:

          mkdir -p /opt/var/spool/cron/crontabs
          ln -s /opt/var/spool /var/spool
          crontab -e

          That’ll setup the required cron directories and get you editing the crontab.

          Cheers,

          Aaron

  2. Stu

    Hi, followed your tutorial and went well until trying to setup / run the on job. I have set it up as described but it doesnt seem to run for some reason. The only thing can think of is not making files read / writtable. Forgive my nievety, i am whats commonly nown as a N00b.

    • Aaron (Author)

      Hi Stu,

      I’m sure we can work it out :)

      First thing to check is that Cron is definitely running correctly. Edit the transmission-scheduler.sh script, and directly under “#!/bin/sh” add:

      touch /opt/bin/test.txt

      And save the file. If the cronjob runs (at all) then an empty file should be created at /opt/bin/test.txt. It’ll obviously run on the schedule you’ve set, so you will need to give the job some time to run (or edit the cronjob schedule to run more often).

      Let me know how it goes. If that file is eventually created, then it’s probably a problem elsewhere in the script.

      Cheers,

      Aaron

      P.S apologies for spelling mistakes/brevity, writing this from my phone :)

      • stu

        Hi Aaron, thanks for the reply. I gave it a shot but stick with me (like i said i am a total N00b). I edited the scheduler script and gave it 5 to run. Then i shh back in and typed /opt/bin # ls and this was the output.

        captoinfo rpcclient
        clear rsync
        crontab smbcacls
        curl smbclient
        curl-config smbcontrol
        event_rpcgen.py smbmnt
        find smbmount
        findutils-find smbpasswd
        findutils-xargs smbspool
        fuser smbstatus
        infocmp smbtar
        infotocap smbumount
        ipkg tdbbackup
        ipkg-opt testparm
        killall testprns
        make_printerdef tic
        make_smbcodepage toe
        make_unicodemap tput
        nano transmission-cli
        ncurses-clear transmission-create
        ncurses5-config transmission-daemon
        nmblookup transmission-edit
        openssl transmission-remote
        peekfd transmission-scheduler.sh
        pidof transmission-sheduler.sh
        prtstat transmission-show
        psmisc-killall tset
        pstree update-alternatives
        pstree.x11 wget
        reset xargs
        rnano

        I figured the script didnt run as there is no test.txt in there. Not sure if i did what you explained correctly but i gave it a shot. Where from here?.

  3. Stu

    That last post didn’t copy properly, the full output was.

    find smbmount
    findutils-find smbpasswd
    findutils-xargs smbspool
    fuser smbstatus
    infocmp smbtar
    infotocap smbumount
    ipkg tdbbackup
    ipkg-opt testparm
    killall testprns
    make_printerdef tic
    make_smbcodepage toe
    make_unicodemap tput
    nano transmission-cli
    ncurses-clear transmission-create
    ncurses5-config transmission-daemon
    nmblookup transmission-edit
    openssl transmission-remote
    peekfd transmission-scheduler.sh
    pidof transmission-sheduler.sh
    prtstat transmission-show
    psmisc-killall tset
    pstree update-alternatives
    pstree.x11 wget
    reset xargs
    /opt/bin #

  4. Stu

    Please let me know if i am going about checking the job is running the wrong way.

    • Aaron (Author)

      Hi Stu,

      Apologies for the delay. Let’s check that the Cron process is running. Run the following command in your terminal and post the output please:

      ps | grep cron

      Aaron :)

  5. Stu

    No worries, ran the command and the said output was.

    ~ # ps | grep cron
    6981 root 2220 S crond
    7850 root 2220 S crond
    8653 root 1696 S /opt/sbin/cron
    9418 root 2220 S busybox crond
    9787 root 2220 S busybox crond
    11770 root 2220 S busybox crond
    17076 root 2220 S busybox crond
    22219 root 2220 S busybox crond
    ~ #
    Getting into this little box of tricks but im struggling with the learning curve.

    • Aaron (Author)

      Hi Stu,

      Looks like you’ve got multiple instances of Cron running. You probably want to kill all of them and then start the service again.

      Run “kill xxxx” where xxxx is the process id (pid) in the left column of the output you supplied.

      Once you’ve cleared up the list, try starting it again (by running “busybox crond”) and check the ps output again. Mine looks like this (just a single instance):

      -bash-3.2# ps | grep cron
      1372 root 3400 S busybox crond
      26939 root 3404 R grep cron

      Let me know how that goes.

      Regards,

      Aaron

  6. Stu

    Job done, killed them all and restarted. This is the output.

    ~ # busybox crond
    ~ # ps | grep cron
    24875 root 2220 S busybox crond
    24960 root 2224 S grep cron
    ~ #

    All those cron jobs were set up by me? (unintentionally).

  7. stu

    I take it the cron job I set up is still in there, just not active. So would I begin by re starting it or make up a new job?. Forgive my persistance but I am determined to make it work.

    • Aaron (Author)

      Hi Stu,

      I emailed you after your last message, did you not get it? Leave me your email address via the contact form on my site and we’ll work through this over email (and hopefully post a solution back here!).

      Cheers,

      Aaron

  8. Quinn

    Hi Aaron,

    Thanks for these walkthroughs! I got my pogo plug up and running this week with transmission and I love it.

    How about a guide to getting the pogo plug to transcode AVI files to MP4s for iTunes? Would be nice to have it download the torrent and then transcode the video files. First big step would just be getting the transcode to work.

    Thanks

    -Quinn

    • Aaron (Author)

      Hi Quinn,

      Thanks for the response! Did you manage to get the steps in this current post working too?

      I can certainly look into automatically transcoding AVI to MP4; I know that ffmpeg is already available as a package via IPKG, but I think it’s going to be an old version that probably doesn’t support MP4 (correctly). One to investigate…

      Cheers,

      Aaron

  9. Quinn

    Hi Aaron, I was able to get this to work as well. I took the suggestion of Andy Bailey above to have transmission watch the folder for me. I also set up a way for me to feed magnet links through a text file in the pogo plug share and used a cron job to run that. Its not perfect but it gets the job done when its hard to get an actual torrent file.

    I messed with ffmpeg but couldn’t get it to work well. I was missing audio and the video didn’t load properly into iTunes. May have been a codec issue. It would be ideal if I could get HandBrake’s command line interface working but that is beyond my skills, if its even possible.

  10. Hi,

    I’m just wondering if you are able to do this and the web server on the same pogoplug?

    Thanks.

    • Aaron (Author)

      Hi Kailan,

      I don’t have this setup with the web server; however I do have Samba running alongside Transmission just fine. My guess is that they should work side by side, but you’d have to try it out :)

      Let me know how it goes.

      Cheers,

      Aaron

  11. This is a functionality I’d love to add to transmission. I’ve got transmission working great and worked through your tutorial step by step. I’ve got the correct IP and torrent paths setup, but my torrent files are not being started automatically.

    I tried to use vi to get back into the script to double check everything, but I’m not able to use vi /opt/bin/transmission-scheduler.sh to get access. I get the following error.

    Error: invalid or corrupt torrent file
    rm: cannot remove ‘/tmp/.cemnt/mnt_sdb1/torrents/*.torrent’: No such file or directory

    Major props for what you’ve helped me accomplish with my Pogoplug so far. I’m very excited at all the potential this little $30 box has to offer.

  12. Jo-An

    Hi Aaron.

    Love your work. I’m a little new to working in ssh. But i seem to be doing allright.
    I have a problem following your instructions.

    Every thing whent fine ( i think) until i have to run “crontab -e”.
    I get the following output:
    Pogoplug:~#crontab -e
    -sh: crontab: not found

    When i go on to the nexst step, setting te intervaltime. I get the following output (witch makes sens, but i still tried it).
    Pogoplug:~#*/5 * * * * /opt/bin/transmission-scheduler.sh
    -sh: */5: not found

    Could you help me out?

    ThanQ Jo-An

    • Aaron (Author)

      Hi Jo-An,

      I have heard of a similar issue from others (via email). Please can you run the following in a terminal and let me know what the output is:

      busybox | grep “BusyBox v”

      Thanks,

      Aaron

    • Servet

      I have the same problem. My BusyBox version is 1.16.1. Is there any solution?
      Thanks

  13. Jo-An

    Hi Aaron,

    Thanks for responding. i’ve run the command.
    Output:

    Pogoplug:~#busybox | grep “BusyBox v”
    BusyBox v1.16.1 (2010-12-01 15:27:23 PST) multi-call binary.

  14. Awesome job, Aaron, I’m so excited I got this working.
    For what it’s worth, I only managed to get this working with an EXT3 formatted USB drive (my first attempt with an NTFS formatted one failed, but there might be another reason)

  15. Pieter

    Hi,

    Thanks for the nice tutorial. I had it working, set up cron to check every minute, and also enabled the watch dir thing, just to be sure.

    When I add the torrent, the torrent suddenly dissapears from the folder, but nothing happens in transmission.
    I can’t seem to figure out what’s going wrong :(

    • Aaron (Author)

      Hey Pieter,

      Have you tried manually starting Torrents from the command line? That’s what the script will be trying to do.

      If you run the following in a console:

      /opt/bin/transmission-remote YOUR_TRANSMISSION_URL -a PATH_TO_YOUR_TORRENT_FILE

      (replacing the two variables!), what do you see?

      Aaron :)

      • Pieter

        Hi Aaron,

        It was due to me using corrupt torrent files. It all seems to be working right now :) Brilliant tutorial by the way!

        I once had to restart the pogoplug, which kind of forced me to re-configure everything. Is there a workaround for that? Also, the drives being mounted in /tmp/ mean that if I unplug one, it’s not sure to receive the same link?

        f.e.: have usb stick in there, in the sdc_1. Have external drive, mounted in sdb_1. If I unplug both & replug them, the usb stick could be on the sdb, rendering all my configuration useless, am I right?

        Poorly formulated, but I hope it’s clear enough. Basically I’m wondering if I can store everything transmission-related on the external harddrive.

      • Brett

        wired-creative-commons-cd.torrent
        It doesn’t work for me, when entering the command manually, I get:

        Pogoplug$ /opt/bin/transmission-remote http://192.168.1.100:9091/transmission/web -a /mnt/disk1/www/transmission/torrents/wired-creative-commons-cd.torrent
        Unexpected response: 405: Method Not Allowed@

        • Brett

          There are html header 1 tags surrounding the “405:Method Not Allowed” in the output, which are removed from the above post.

          • Aaron (Author)

            Hi Brett,

            What happens if you try loading the torrent (or a different torrent) from it’s URL (instead of from the filesystem?).

            Could the local torrent file be corrupt/malformed at all?

            Aaron

  16. Brett

    I can load the above torrent into the web interface without issue. Is this what you mean, or will transmission-remote take an url as input after the -a switch?

    It is something that the transmission-remote utility does not like. I got tired last night and forgot to take some notes, I will retry some things and update.

    Thanks for the quick reply and the excellent tutorials!
    Brett

  17. Brett

    I ran the above command with the -b switch, getting something like
    “unknown option (url ip address)”
    So I modified your transmissions scheduler script to remove any reference to the url.
    transmission-remote will be running on the same system as the daemon, so the url option is redundant anyways.
    Works well now, thanks.

    Brett

  18. Jamie

    Hey there,

    Thank you for this tutorial. The torrents seem to disappear from the folder but don’t get added to Transmission. Pl. help.
    (Read through the comments section and found one that was faced with a similar predicament)

    If you run the following in a console:
    /opt/bin/transmission-remote YOUR_TRANSMISSION_URL -a PATH_TO_YOUR_TORRENT_FILE

    Pogoplug:~$ /opt/bin/transmission-remote http://192.168.1.3:9091/transmission /tmp/.cemnt/mnt_sdb1/Torrents/Hulk_2003_1080p.torrent
    Unknown option: /tmp/.cemnt/mnt_sdb1/Torrents/Hulk_2003_1080p.torrent
    Pogoplug:~$ /opt/bin/transmission-remote http://192.168.1.3:9091/transmission /tmp/.cemnt/mnt_sdb1/Torrents/
    Unknown option: /tmp/.cemnt/mnt_sdb1/Torrents/
    Pogoplug:~$ df -h
    Filesystem Size Used Available Use% Mounted on
    /dev/mtdblock2 32.0M 26.7M 5.3M 84% /
    none 61.6M 500.0k 61.1M 1% /tmp
    /tmp/.cemnt/sda1 931.5G 5.1G 926.4G 1% /tmp/.cemnt/mnt_sda1
    /tmp/.cemnt/sdb1 931.5G 185.5G 746.0G 20% /tmp/.cemnt/mnt_sdb1
    Pogoplug:~$

    Thanks.
    Jamie.

  19. Jason

    Hey Aaron,

    Inspired by you, set out to write my own lil dirty script to pull magnets from a txt file as well.

    Maybe it will help someone, I haven’t coded in bash, ever! (More of a SQL and Oracle ADF fusion guy ;)

    Anywho, this works:

    The base steps are as in this post, all you have to do is modify the “transmission-scheduler.sh” script with the code below
    ———————————————————
    #!/bin/sh
    pathToTorrentFolder=’/tmp/.cemnt/mnt_sdb1/Torrents/’
    transmissionUrl=’http://192.168.1.3:9091/transmission’
    pathToMagnetFile=’/tmp/.cemnt/mnt_sdb1/Torrents/magnets.txt’
    tr_options=”192.168.1.3″

    if [ "$(ls -a $pathToTorrentFolder)" ]; then
    cat $pathToMagnetFile |
    while read m
    do /opt/bin/transmission-remote $tr_options –add “$m”;
    rm “$pathToMagnetFile”
    done
    fi
    if [ "$(ls -a $pathToTorrentFolder)" ]; then
    for f in $pathToTorrentFolder*.torrent ;
    do /opt/bin/transmission-remote $transmissionUrl -a “$f”; rm “$f”
    done
    fi
    ———————————————————

    If any “bash-ful” person can clean this up, thanks!

    Cheers!
    Jason

  20. Cport89

    Hi,
    First of all Thank you for your posts about customizing our pogoplug with Things like optware;-)

    I would like to know:
    Is it posible to Auto unzip/unrar Files after downloading them with transmission?
    I found an Solution via the settings.json, but it unrars only small Files out of the archive and Not all of them…

    Maybe you have a similiar Solution like for adding torrents?

    Kind regards and sorry for my bad english

  21. Andy Bailey

    Hi Aaron.

    I’ve been using transmission on my Pogoplug for a couple of years now after following your instructions. I’ve recent started to notice however that it seems to struggle with larger downloads – I’ve tried to get it to download a .torrent from the TOSEC site that’s around 4GB, and it really seems to get confused by reporting impossible download speeds and not throttling (or at least that’s what it _appears_ to be doing).

    So, I was wondering if you knew of any download packages where you can feed it a URL to a file (such as a .zip file rather than a torrent) and it does all the work in the background, no need to use a “full” computer and leave it on?

    I thought that something like mldonkey might do the job, and I’ve tried to muddle my way through downloading the package using ipkg, but all I seem to end up with is a set of files of varying sizes with a .byte extension such as mldonkey.byte

    As I’ve no idea what I should really be doing, I wondered if this might be something you could cover at some point?

    Andy