Rclone Dry Run: What You Need to Know

A dry run with Rclone is a great way to test your backup or sync script without actually transferring any files. This feature is super useful for catching errors and making sure everything is set up correctly.

Rclone's dry run mode is enabled by adding the `--dry-run` flag to your command. This flag tells Rclone to simulate the transfer without actually moving any files.

You can also use the `--stats` flag in conjunction with `--dry-run` to get detailed statistics about what Rclone would have done during the transfer. This can be really helpful for debugging and optimizing your script.

By running a dry run, you can identify and fix issues with your configuration before actually transferring any files.

Rclone is a command-line program that synchronizes files and directories between two locations, such as a local computer and a cloud storage service. It's designed to be efficient and flexible.

Rclone supports over 40 cloud storage services, including Google Drive, Amazon S3, Microsoft OneDrive, and Dropbox, making it a versatile tool for managing your online storage needs.

To use Rclone, you'll need to install it on your computer and create a configuration file that specifies the source and destination of your file sync operation. This file is called a "remote" in Rclone terminology.

Rclone's basic syntax is pretty straightforward. A subcommand is the main operation you want to perform, such as sync or copy.

A subcommand is always required, and it's usually followed by options and parameters. Options can be single letter flags like -v or a group of flags like -Pv, or long flags like --progress.

No options are required, but you can use them to customize your command. Options can come after the subcommand or in between parameters, but only global options can be used before the subcommand.

A parameter is usually a file path or rclone remote, such as /path/to/file or remote:path/to/file. You can specify source and destination paths by using the name you gave the storage system in the config file, followed by a sub path, like "drive:myfolder".

You can define multiple storage paths in the config file, and use them to specify source and destination paths. Just remember to use the --interactive/-i flag while learning rclone to avoid accidental data loss.

Rclone is a versatile tool that enables you to interact with multiple cloud storage services from a Linux command line interface (CLI). It's perfect for transferring files between different cloud providers or syncing your local files with a cloud storage account.

Rclone allows you to sync files and directories between your Linux server and cloud storage platforms like Google Drive, Dropbox, and Amazon S3. This means you can keep all your files up to date across multiple services effortlessly.

With rclone, you can efficiently transfer files to and from cloud storage providers, whether it's a single file or an entire directory. This is especially useful when you need to move large files or multiple files at once.

Here are some of the key features and benefits of rclone:

Rclone Commands are incredibly powerful and flexible. Here are a few key examples of how to use them.

The rclone sync command is a game-changer for synchronizing files and directories. It compares the source and destination locations, intelligently updating the destination to match the source.

You can use the rclone sync command to sync local files with a remote provider. The basic syntax is `rclone --dry-run --progress sync /path/to/local/directory remote:CloudStorageFolder`, which compares and syncs files, ensuring efficient data transfer.

To sync remote cloud storage providers, use the command `rclone sync sourceRemote:SourceFolder destRemote:DestinationFolder`. This command syncs files between two remote cloud storage locations.

When you want to mirror the source location, including deletions, use the command `rclone sync --delete sourceRemote:SourceFolder destRemote:DestinationFolder`. This command is especially useful when you need to keep your remote storage up-to-date.

Here are the basic rclone sync commands summarized:

Rclone has a number of options to control its behavior, including options that can be set to true or false, which behave slightly differently than other options.

To set options, you can use the --option=value or --option value format, or for boolean options, you can simply use --boolean to set it to true, or omit the flag to set it to false.

Note that semicolons can be used as separators instead of spaces to avoid parsing issues in environments like Docker.

Here are some common options and their environment variable equivalents:

The options set by environment variables can be seen with the -vv flag, e.g. rclone version -vv.

Rclone has a number of options to control its behavior.

Boolean options behave slightly differently, setting the option to true with --boolean and false without the flag. You can also specify --boolean=true or --boolean=false, but not --boolean false.

Semicolons can be used as separators instead of spaces to avoid parsing issues in environments like Docker.

If a schedule is provided, the file will use the schedule in effect at the start of the transfer.

Boolean options include --boolean, which can be set to true or false. To set it to false, simply omit the flag.

Semicolons can be used as separators, but spaces are also acceptable.

Rclone can limit bandwidth with the --bwlimit option, which can be specified in various ways. For example, --bwlimit=10M would limit both upload and download bandwidth to 10 MiB/s.

The bandwidth limit can also be specified separately, such as --bwlimit=10M:100K, which would limit upload to 10 MiB/s and download to 100 KiB/s. To disable a limit, use "off", like --bwlimit=10M:off.

A timetable of limits can also be specified, using the format WEEKDAY-HH:MM,BANDWIDTH. For example, --bwlimit "08:00,512k 12:00,10M 13:00,512k 18:00,30M 23:00,off" would set the bandwidth to 512 KiB/s at 8am, 10 MiB/s at noon, and 30 MiB/s at 6pm.

The units of bandwidth are bytes per second, not bits per second. To convert from bits per second, divide by 8.

Environment variables can be used to set defaults for Rclone options, making it easier to configure the tool without having to specify every option on the command line. This can be a huge time-saver, especially for complex commands.

To set an environment variable, you need to take the long option name, strip the leading '--', change '-' to '_', make it uppercase, and prepend 'RCLONE_'. For example, to always set '--stats 5s', you would set the environment variable 'RCLONE_STATS=5s'.

The environment variable equivalent of '--verbose' or '-v' is 'RCLONE_VERBOSE=1', or for '-vv', it's 'RCLONE_VERBOSE=2'. This means you can easily increase the verbosity of Rclone by setting this variable.

You can see the options set by environment variables by running Rclone with the '-vv' flag. This is a great way to check what options are being set and how they're being applied.

If you have options that can appear multiple times, such as '--exclude', you can treat the environment variable as a CSV-encoded string. For example, you can set 'RCLONE_EXCLUDE="*.jpg,*.png"' to exclude both '.jpg' and '.png' files.

If you're not a fan of GUIs, you can use the rclone CLI to upload and download files. The CLI commands are familiar to those who navigate a bash terminal.

You can use the following commands to get started: rclone config — used to run an interactive session for setting up a remoterclone copy — used to copy files to a remote but ignores already existing files. Used for directoriesrclone copyto — used to copy individual filesrclone lsd — used to list directories on a remoterclone ls — used to list files on a remoterclone tree — tree view of the filesrclone mount — used to mount a remote as a part of your filesystem

For example, copying a folder to a particular path in a bucket on your backblaze r2 remote would look like rclone copy /path/to/local/directory remote:CloudStorageFolder.

The Rclone CLI is a powerful tool for managing your files and directories. It's a great alternative to the GUI if you're comfortable navigating a bash terminal.

You can use the rclone CLI to upload and download files, and it has a range of useful commands to get you started. Some of these include rclone config, rclone copy, rclone copyto, rclone lsd, rclone ls, rclone tree, and rclone mount.

One of the key features of the rclone CLI is its ability to mount a remote as a part of your filesystem. This allows you to access your remote files as if they were local. However, this is not supported out of the gate on Macs and requires additional configuration.

You can list directories on a remote using the rclone lsd command, and list files on a remote using the rclone ls command. The rclone tree command provides a tree view of the files, which can be helpful for navigating complex directories.

Here are some examples of how to use the rclone CLI to copy files and directories:

The rclone CLI also has a range of subcommands that can be used to perform specific tasks. Some of the most commonly used subcommands include rclone config, rclone copy, rclone sync, rclone bisync, rclone move, rclone delete, and rclone purge.

Some commonly used flags for the rclone CLI include --dry-run, --progress, --delete, --transfers, and --checkers. The --dry-run flag simulates the command without making any changes, while the --progress flag shows progress during file transfers.

The rclone sync command is a powerful tool for synchronizing files and directories. It compares the source and destination locations, intelligently updating the destination to match the source. Here are some examples of how to use the rclone sync command:

The Dump Filters feature in Rclone CLI is a game-changer for anyone who needs to troubleshoot their filter settings. It allows you to see exactly what include and exclude options are filtering on.

This is especially useful if you're having trouble figuring out why certain files aren't being synced or transferred correctly. The command "--dump filters" does just that, dumping the filters to the output for your reference.

With this feature, you can easily identify any issues with your filter settings and make the necessary adjustments to get your sync or transfer back on track.

Before running a rclone command, it's a good idea to check the data integrity of your files. This can be done with the --fast-list option, which enhances the speed of the integrity checks.

Using the --fast-list option is a must for all data integrity check commands. This ensures that your checks are as efficient as possible.

You can test the integrity of transferred data with rclone by checking the checksums on both the source and target sides. This is a great way to verify that your files are safe and intact.

For fast checks, you can use the --size-only option, which only checks file sizes. This can save you time and effort.

If you're using an encrypted bucket, you'll need to use the cryptcheck option to check data integrity. Be aware that using the check option on an encrypted volume will force the download of all data in the checked path, which can be unnecessary and stall your client.