20
Solving Eric's storage problems
Welcome back! 👋🏻 This time is not a development post, but it's something quite interesting if you need more space o a good backup strategy.
Eric is a good friend of mine, we've worked together in Globant around 2013 and we've worked on a freelance project for an E-Commerce site. He is a gifted graphic designer, photographer, and musician.
He had some trouble with his computer and replaced it with a new one. Having the old hard drives working fine, he wanted to use them as external hard drives for backup.
That's not bad, not at all, one backup is great if you had none, but we all know that a single backup is no backup at all.
When you work with clients' assets you shouldn't rely only on an external hard drive. This is way too risky, and it wouldn't be nice to ask for the assets again because your hard drive failed.
Some people have two+ physical hard drives or save the assets on their Gmail account but they are not very convenient or they can become very expensive, very soon.
Well, you can have multiple accounts and sparse the assets between all the accounts, but it means that you should know where you've saved what, and keep logging in and out can be quite time-consuming.
Having the data on the cloud is definitively the best place to have the backup of his assets, but it can be hard to manage if he doesn't organize them.
Also, there is a problem with having multiple desktop clients for each cloud provider. They consume a lot of resources that he is going to need to work on his computer.
Here comes RClone They call it "The Swiss army knife of cloud storage" and I agree.
It allows him to back up, restore and sync data from his computer to almost any cloud provider. Mount cloud drives as network-attached / local disks. Check files for integrity and also union file systems to present multiple disks as one.
Now that we know the tool, let's kick the tires and see how it works. 💨
Let's suppose he wants to enjoy having 100GB for free courtesy of MEGA.io so he has to create two accounts. I won't get into the details, it's just filling a couple of forms and confirming emails.
Once he has both accounts, we can set up RClone.
RClone is a binary made in go. It means that it only runs when we execute it but we wanted to have it running all the time, like a service.
To run RClone as a service we can use an application called NSSM (Non-Sucking Service Manager) I suggest downloading the "latest release" version. It says on the NSSM webpage: "No "installation" of nssm is needed. Just place it somewhere on the system (preferably somewhere in your PATH) and run it."
Also, on Windows, we need to install winfsp to make possible mounting remote folders.
The last step to run RClone is to configure the remote (each mega account)
To do that, we open a command prompt and run "rclone config" and go through these steps for each remote:
- "New remote" - Type name of the remote (this is not the provider's name, but a unique name to identify this remote drive)
- Select the provider (here we can use Box, Dropbox, OneDrive, GDrive, but in this case, we are going to use Mega)
- Type the username (mega's account username)
- Type the password (and confirm)
- "Edit advance config?" (no, at this time)
- Confirm that everything is OK.
Let's verify that we can access our Cloud Drive. Open a command prompt and type "rclone mount --vfs-cache-mode full {remote name1)}:/ {drive letter}: {remote name2)}:/ {drive letter}:"
Where "remote name" is what you typed in step 2 of the last section and "drive letter" it's the letter that you want to use to access it in My Computer.
A couple of issues we may face in this step. ⚠
- I get this error: "2021/05/14 09:48:32 Fatal error: failed to mount FUSE fs: mount stopped before calling Init: mount failed: cgofuse: cannot find winfsp" - This means we haven't installed WinFsp, which is needed for mounting in Windows.
- I run the command, it says: "The service rclone has been started." but I don't see the drive in My Computer. - This happens when the command is executed on a command prompt with elevated privileges (as Administrator). The command must be executed by the user who wants to have the drive mounted.
Well, we've installed the programs, configured the remote, and tested that it mounts as a drive on our computer. Right now it will be available while the RClone command is running and as soon as we close it, the drive will disappear from our computer. To make these changes permanent, we must run RClone as a service, and here is where NSSM comes in.
To create a service for RClone we ran the command "nssm install {service name}"
Where "service name" is how we want to identify the service that will be running RClone for us.
In the application tab, we must fill in the path of our RClone binary location. Here is a tip, if you don't know where it is, you can find it by typing RClone in your start menu and then look for "Open file location", and then the arguments which are everything we typed after the "rclone" command on our testing. "mount --vfs-cache-mode full {remote name)}:/ {drive letter}:" (Refer to "Test run mounting your cloud drive" to know what "remote name" and "drive letter" mean)
Then we go to the "Log on" tab, select "This account..." and fill in your Windows user credentials. Another tip, if you don't know your FULL windows username you can open a command prompt and execute "whoami" and it will get you the information you need in NSSM.
Finally, we must go to the "Shutdown" tab and change the timeout for "Generate Control-C" to 10 seconds, it is needed because if RClone is syncing data, it won't exit immediately and if we force it, it may corrupt data. And then click "Install service"
We have two options, the first one is to reboot the computer to have the RClone service started or we can initialize the service manually on Task Manager. To do that, we must open Task manager and look for the "Service" tab, there we should find our fresh service waiting to be started. Right-click it, and then click "Start".
And now we have our Cloud Drive ready to go. 🚀
Well, first of all... you suck! But I understand, sometimes having a graphical interface saves quite some time searching for the correct argument.
That's why RClone has a great Web GUI where you can tweak everything related to any remote. Just as an example, let's set a bandwidth limit because, as it is configured, it will use every single drop of the bandwidth you have (not cool)
To open the Web Gui, we have to open a command prompt and execute this command: "rclone rcd --rc-web-gui" It will open a browser window with a dashboard where we can tweak RClone.
On your right, you will find "Bandwidth". Click on "Modify". As an example, I set 300Kb for uploads and 1Mb for downloads.
But there are many options you can change, add remotes, setting mounts. I highly recommend going through the RClone documentation
Syncing data takes time, much more time than copying files to a hard drive or USB Stick. When you copy a file to a Cloud Drive, it will look like the copy is finished, but actually, RClone is syncing in the background.
This is meant to be used as a backup drive. If you need to upload a file immediately, you can use RClone web GUI or each service website.
If you've got here, Thanks!!! It's quite a load of information to process, so I appreciate you've read it. If you've found this information useful or interesting, please let me know in the comments, thoughts, and if this topic gets enough attention I'll write about how you can backup USB drives automagically or how to "kind of" fusion your cloud drives for "unlimited" storage. 🤯
20