Sunday, December 6, 2009

Here's what I've got

Like I said, the original program was a humnble bash script. I can post it if there is any interest, but will otherwise move forward.

Part 1.
I wrote a script in Perl that will export meta information from all torrents running in Transmission to csv format.

Here is the new export script for Transmission: http://pastebin.ca/1704262

Note: fields are separated by ";", if any semicolons appear in your filenames, you'll have to remove them by hand before importing to mysql.

Part 2.
I will begin Part 2 this week, creating a script to export the date added, and the amount uploaded and downloaded. This will be used in the second table that measures daily change in UL and DL. This part should be simple, I just need to modify the first script a little bit to look for UL and DL.

Part 3.
It doesn't make sense to reimport the same information every time one or two torrents are added, so I will modify the original scripts to only import changes.

These three portions are unique to Transmission and not of much use to anybody using a different client. However, the next part will be useful.

Part 4.
This is the meat of the program. It contains the web interface, which allows the user to view statistics and make manual changes to each torrent.

It also measures torrent performance, selects torrents for deletion, and adds new torrents as space allows.

I intend for this section to be client neutral, meaning that it won't interact with the client directly, it will just call other scripts to add or delete as needed.

Though the previous part is written in Perl, I will write Part 4 in PHP, mostly because I need to practice for work.

In the beginning

This project began as a mere shell script that pulled information from transmission-remote to measure torrent performance.

Users set the maximum total size for all torrents, and once that limit is reached, the lowest performing torrents are removed. Torrents can be whitelisted, making them ineligible for deletion, and a minimum seeding duration can be set to ensure the performance averages are accurate.

The benefits of this tool are:

1. Efficient distribution of seeders. Additional seeders on popular torrents upload very little, their disk space is better used on 'rarer' files. By measuring torrent performance as ratio/time (total uploaded / size / time seeded), users only seed torrents that need seeders. This will increase torrent life, and user upload.

The new version add functionality to the current shell script and can be used by clients other than Transmission, so long as somebody develops the software to export from their client of choice.

First, torrent meta information is stored in a mysql table:
1. torrent name
2. hash
3. format
4. bitrate
5. torrent creation date (not the date the torrent was added)
6. date deleted
7. whitelisted (Y/N)
8. converted to other format (Y/N)
9. size
10. location on drive

Each torrent is also given an ID number (because hashes are not human friendly).
This ID links the torrent to another table where the amount uploaded and downloaded is recorded for each day.

Enough about data storage, what can we do?

Assuming your drive is maxed out (or whatever size limit you've set)
torrentPrune will look for the lowest X% performing torrents and delete those that are eligible, then it will add new torrents from a specified directory in their place.

You can also schedule a script to run after a torrent completes. For instance, if the added torrent is FLAC, then you can automatically convert it to V0 and create a new torrent once finished.

That is the core functionality, automating the management of your torrents. There will also be a web interface for manual deletion, setting whitelist status, etc.

There are many possibilities beyond this main objective. sql driven media management, etc