关于Rclone

Rclone是一个基于命令行管理云存储的软件。它功能丰富，可以在一定程度上替代云服务商自带的Web管理接口。Rclone支持连接超过40家云服务商，其中包括S3类型的对象存储，商业和消费级的文件存储服务，以及标准传输协议（Webdav，FTP等）。

Rclone可以做什么？

  • 将文件备份（以及加密）到云端
  • 从云端下载（以及解密）文件
  • 迁移云端数据
  • 挂载云端空间为本地磁盘

本教程主要讲解如何使用rclone在Linux（Ubuntu 20.04）下备份文件。

安装Rclone

前往Rclone官方网站手动下载安装包，或运行下列命令

curl https://rclone.org/install.sh | sudo bash

配置Rclone

接下来，我们使用Rclone连接云服务商（这里以Google Drive为例），其他云服务商的连接方式会略有不同，请参考官方文档

rclone config
No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q>
  1. 输入n，连接一个新云盘
name>
  1. 为云盘起一个名字（随意起）
·····
·····
12 / Dropbox
   \ (dropbox)
13 / Encrypt/Decrypt a remote
   \ (crypt)
14 / Enterprise File Fabric
   \ (filefabric)
15 / FTP Connection
   \ (ftp)
16 / Google Cloud Storage (this is not Google Drive)
   \ (google cloud storage)
17 / Google Drive
   \ (drive)
18 / Google Photos
   \ (google photos)
19 / Hadoop distributed file system
   \ (hdfs)
·····
·····
Storage>
  1. 输入17选择Google Drive（注意：序号可能会变化）
Option client_id.
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a value. Press Enter to leave empty.
client_id>
  1. 这里提示生成我们自己client_id可以提升备份效率，因为我们只是备份小文件，因此留空。
Option client_secret.
OAuth Client Secret.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_secret>
  1. 继续留空
Option scope.
Scope that rclone should use when requesting access from drive.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
 1 / Full access all files, excluding Application Data Folder.
   \ (drive)
 2 / Read-only access to file metadata and file contents.
   \ (drive.readonly)
   / Access to files created by rclone only.
 3 | These are visible in the drive website.
   | File authorization is revoked when the user deauthorizes the app.
   \ (drive.file)
   / Allows read and write access to the Application Data folder.
 4 | This is not visible in the drive website.
   \ (drive.appfolder)
   / Allows read-only access to file metadata but
 5 | does not allow any access to read or download file content.
   \ (drive.metadata.readonly)
scope>
  1. Rclone询问需要怎样的权限。1权限可以读+写Google Drive上的所有内容（慎重选择）；2权限可以读Google Drive上的所有内容；3权限可以读+写Rclone在Google Drive上自己创建的文件；4、5应该用不到。

我们这里选择3足以（备份不需要Rclone可以访问Google Drive中已有的文件）

Option root_folder_id.
ID of the root folder.
Leave blank normally.
Fill in to access "Computers" folders (see docs), or for rclone to use
a non root folder as its starting point.
Enter a value. Press Enter to leave empty.
root_folder_id>
  1. 这里默认留空
Option service_account_file.
Service Account Credentials JSON file path.
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.
Enter a value. Press Enter to leave empty.
service_account_file>
  1. 这里继续默认留空
Edit advanced config?
y) Yes
n) No (default)
y/n>
  1. 这里选择n
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine

y) Yes (default)
n) No
y/n>
  1. 如果你使用的是Linux命令行（没有桌面），这里务必选择n；如果你有桌面环境，请选择y，之后会在浏览器弹出一个窗口登录Google Drive。

由于我使用的是命令行版本，因此我选择n

Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
        rclone authorize "drive" "XXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
Then paste the result.
Enter a value.
config_token>
  1. 接下来，在另一台有桌面环境的主机（需要安装Rclone）输入
rclone authorize "drive" "XXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
  1. 浏览器会自动弹出
2022/05/24 12:05:11 NOTICE: If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=oXaZl7Y2-059s1y72cSXbQ
2022/05/24 12:05:11 NOTICE: Log in and authorize rclone for access
2022/05/24 12:05:11 NOTICE: Waiting for code...
2022/05/24 12:05:15 NOTICE: Got code
Paste the following into your remote machine --->
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
<---End paste
  1. 完成验证后，会有Success!的提示
config_token> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  1. 将Token粘贴到原主机
Configure this as a Shared Drive (Team Drive)?

y) Yes
n) No (default)
y/n>
  1. 个人版Google Drive选择n
[Name]
type = drive
scope = drive.file
token = {"access_token":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX","token_type":"XXXX","refresh_token":"XXXXXX","expiry":"XXXXXX"}
team_drive = 
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d>
  1. 选择y
Current remotes:

Name                 Type
====                 ====
Name                 drive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q>
  1. 选择q退出
  2. 配置完成

上传测试

首先查看一下Rclone能否获取到我们的任何文件

rclone ls (你Remote的名字，步骤2你起的名字):path

例如：

rclone ls Name:/

由于我们之前只允许Rclone读取它自己创建的文件，因此里面应该是空的（除非之前用过Rclone）

现在，我们来新建一个文件夹

rclone mkdir (你Remote的名字，步骤2你起的名字):path

例如在根目录新建一个名为test的文件夹

rclone mkdir Name:/test

此时，Google Drive网页版上如果多出一个test的文件夹，代表设置成功！

切记不要在Google Drive网页版直接新建文件/文件夹，由于这个文件/文件夹是你（Owner）新建的，因此Rclone是读取不到的！（除非你在设置的第6步选择了1.Full Access）

使用shell脚本备份并上传

通常来说，我们需要备份服务的文件夹以及数据库，下列脚本仅供参考

#!/bin/bash
## 零点执行该脚本

MYSQL_PWD='DATABASE PASSWORD'
WEB_NAME='APPLICATION NAME'
DB_NAME='DATABASE NAME'
RCLONE_PATH='{RCLONE REMOTE NAME}:{REMOTE PATH}'


## 获取当前时间
TIME=$(date "+%Y-%m-%d_%H-%M-%S_UTC%z")
TIMEYY=$(date "+%Y")
TIMEMM=$(date "+%m")
## 数据库备份
mkdir -p /tmp/backup${WEB_NAME}
cd /tmp/backup${WEB_NAME}
mysqldump -h127.0.0.1 -uroot -p${MYSQL_PWD} --default-character-set=utf8mb4 "${DB_NAME}" > "${DB_NAME}.sql"
## 根目录打包压缩
cd /home/wwwroot
tar -czvf "/tmp/backup${WEB_NAME}/${DB_NAME}_${TIME}.tar.gz" "./${WEB_NAME}" "/tmp/backup${WEB_NAME}/${DB_NAME}.sql"
## 将文件传送到云端
/usr/bin/rclone copy -P "/tmp/backup${WEB_NAME}/${DB_NAME}_${TIME}.tar.gz" "${RCLONE_PATH}/${TIMEYY}/${TIMEMM}"
## 删除临时文件
rm -rf /tmp/backup${WEB_NAME}/

Rclone复制命令

rclone copy /local/path remote:path # copies /local/path to the remote

使用crontab每晚0点定时运行脚本

crontab -e
0 0 * * * /bin/bash /yourls_backup.sh（你脚本的路径）