Note: This is a project, rather than a tutorial, and such may not be particularly succinct or easy to follow. Please read it all the way through before starting as there may be entire sections that you can skip.
Sub-note: I do not condone piracy. This tutorial series is designed to allow you to download legal movies, TV shows and audiobooks and stream them.
Today, we’ll be putting together a Plex Media Server with an Amazon unlimited size cloud drive.
Cloud Storage used in ways it was not intended
Everybody has heard of and probably used cloud storage. Dropbox, Mega, SkyDrive, whatever. Very recently I found out about Amazon’s contender: Amazon Cloud. I hadn’t heard of it before, probably because it skimmed its way under the radar here in the UK as it doesn’t offer anything particularly crazy. In the US however, they offer an “Unlimited Everything” package, unlimited file storage for only $60 a year.
As a person with near enough 5TB of media and on a network that doesn’t allow port forwarding (Student life!), this caused a pretty severely raised eyebrow. If I could store and access all of my media on the go from a reliable source, not only could I cut down on my drives at home (All of which are fast approaching “Warning” on SMART due to 20k+ hours of uptime), but I would be significantly more comfortable on trains, especially with the PlexSync(assuming I ever actually bother getting a PlexPass). Not to mention it means I could just set up my dedicated box (which is already a torrent server) to upload straight to Amazon and have it accessible to all of my devices.
So let’s go through the steps of trying to get a cloud based media server running.
An Amazon account: Thankfully, if you’re not willing to shell out $60 for an unlimited everything package just yet, Amazon offers 5GB for free for you to play with before you upgrade, so I’ll be using that. This is going to be our cloud storage. Note that this must be an American Amazon account, and a UK account will just redirect to the UK site. Make sure you register and sign up to cloud drive with a US account.
A Dedicated Hosted Server: Yeah, this is where most people fall short. Any VPS or hosted/local server will do as long as you can forward ports and it has a decent download and upload speed. This is going to be the “head” for our storage: We’ll trick it into thinking the Amazon storage is local, then use it to stream data to our clients. Effectively, it’s a proxy. I get most of mine from OVH, and you can get fairly cheap ones from the OVH company Kimsufi. Bear in mind that if you set your Plex to convert your files, you will not only need a fast internet connection to stream it from Amazon, but a fast CPU to convert it on the fly.
Some Linux Knowledge: For this project I’m assuming the dedicated server is an Ubuntu based server. In theory it should work fine with any Debian based distribution, and with some modifications of the programs in use it could even work with Windows. I am by no means a Linux pro, so if you see any improvements to be made, please inform me.
A PLEX account: PLEX is the easiest way I’ve found to quickly stream media to pretty much any platform, so I’ll be using that to try and view my library online.
Where do we start?
Being that this is actually the second iteration of this project, I’ve found much less frustrating ways to get it running. We'll be starting with Plex itself. I quite like putting everything under the "Plex" user for neatness sake, though the Plex user isn't actually accessible by default - We'll fix that later.
First up, let’s grab some Plex and set it up. SSH into your server and get ready for some terminal action. If you’re using Putty, under the “Connection” tab you may want to enable keepalives so we don’t randomly get kicked off while waiting for something.
Setting up Plex
For now, let’s jump into our home directory, download and install plex(Latest version at time of posting). This should be fairly easy.
sudo dpkg -i plexmediaserver_0.9.15.6.1714-7be11e1_amd64.deb
If we navigate to http://serveripaddress:32400/web we should get the Plex login, fantastic! If you like, you can login, but unfortunately we can’t yet configure Plex due to being a client. Before we bother with that, let’s make our Plex user a bit more usable.
First, we need to make plex an account we can login to so we can test stuff out. We then need to make it a sudoer so we can actually do things, and we can also disable its SSH access to make it a little more secure.
adduser plex sudo
This allows us to jump over to the root user and then update the Plex user password, and makes them a sudoer. Let’s disable their SSH login.
Scroll down to the bottom and type
Save, and run
service ssh restart
That should disable the plex account from being logged in via SSH. To get to it, we now need to su over to it. Let’s do that now.
Now, the plex user technically does have a home directory, it’s just in an awkward place. To make life a little bit easier when trying to work with files, let’s just make a folder for plex to play with in the home directory. Note that “~” will NOT resolve to this directory, this is just for playing with.
sudo mkdir /home/plex/
sudo chown plex /home/plex/
Because it will be used later as our cloud drive storage location, let’s just quickly make an amazon folder.
Since we’re in our pretend home directory, we can start setting up acd_cli, which is the application we’ll be using to access our cloud drive.
We really should get some test data on our Amazon Cloud. I used a YouTube music video I had sitting around. You can start uploading your entire media library if you like, but for now I’m being conservative. On a desktop machine, uploading single files is trivial, merely go to your Amazon Cloud Dashboard and drag a video file or two onto the browser window.
Next, we should make our media visible on our dedicated server. For this we’ll be using an open source app called acd_cli by a user on GitHub called yadayada. Effectively, it’s a command line interface that "symlinks" your cloud drive to an accessible part of the filesystem, so you have read/write access.
The easiest way to get acd_cli installed is by starting with easy install from python-setuptools. Let’s grab a bunch of other dependencies at the same time, as well as “pip”, a package manager for python.
sudo apt-get install python3-setuptools python3-appdirs python3-dateutil python3-requests python3-sqlalchemy
sudo easy_install3 -U pip
Once that’s done, we can grab acd_cli from source with all its dependencies
sudo pip3 install –upgrade git+https://github.com/yadayada/acd_cli.git
We can ensure all that worked by running
You should have a request for cookies. Hit A to accept all the incoming cookies. After a few moments you’ll have a fantastic replica of the Amazon website in text.
Since text isn’t my favourite way to browse, and probably isn’t yours either, hit Q then Y to quit, and take note of the link it printed.
On your computer, navigate to https://tensile-runway-92512.appspot.com/ , which will direct you to Amazon to login. This step is to authorise the application to your Amazon account.
After authorising, it should have downloaded a file called “oauth_data” for you. SFTP into your dedicated server and drop this file into
/var/lib/plexmediaserver is the real home directory for plex, by the way). This may take a little bit of effort, since you can’t login as plex and your other user probably doesn’t have permission into Plex’s home folder, but as long as it makes its way into the folder it’s fine. Personally I SFTP'd it into my home directory, then used
sudo mv oauth_data /var/lib/plexmediaserver/.cache/acd_cli/
sudo chown plex:plex /var/lib/plexmediaserver/.cache/acd_cli/oauth_data
Note that if you copy it using root, you’ll need to chown the file to plex so it can read it.
If you exited acd_cli to do the copying, run
acd_cli sync again or just press enter if you hadn’t. It should correctly sync.
Now we can actually mount the drive using acd_cli
acd_cli mount -ao /home/plex/amazon
This may give you a couple of “fusermount” permission errors. This is because plex doesn’t have write permission for a couple of things, but we can actually ignore this error and cd into our amazon folder, where we should now have a mounted drive! If you're wondering, the
ao means "allow other users". This is insanely handy later.
So here is where we reach our first caveat - Yes, you have mounted your Amazon drive, and you can even read and write files to it and check out how cool it looks in the web UI. Unfortunately, large files don't work so well.
Thankfully, we have a fix for that later.
Since we’re not local to our Plex installation, Plex web isn’t going to give us half the settings we want. We can make ourselves tunnelled so Plex thinks we’re local using Putty.
Open up Putty, enter all your normal connection settings, then go down to the SSH menu, and open Tunnels. In source port, enter
32400 and in destination enter
localhost:32400. Once you login, you should have a working tunnel.
In your browser, navigate to http://localhost:32400/web . You may need to log in again, but once you do you can click the settings button in the top right. Set it up how you like. You may need to add your login once more, You may also want to tick “Update my library automatically”.
Once done, hit the home button at the top left and find your server in the navigation pane on the left. Click the little + button to add a new library. In my case, I knew my test video's name was formatted weirdly so I picked Home Video, and added Amazon as a library. Plex will do a quick scan and then huzzah, you should be able to stream media!
Note that if you have long loading times you may have left transcoding on. On a slow server, transcoding will take a long time.
If you add more media to cloud drive, you will have to manually run “acd_cli sync” to update your cloud drive, then do a library refresh on Plex. We will automate this in future.
In part two, we explore adding a torrent server into the mix, automatically uploading the torrents into the cloud, and deleting them from the local server.
In parts three and four, we’ll explore automated ways of grabbing torrents, automatically uploading into different folders (and different libraries) for types of media, setting up crons for syncing, and boot-time mounting and starting of scripts.