Rclone MacOS Setup for Cloud File Access

Rclone is a powerful tool for accessing cloud storage on your Mac. It supports over 40 cloud storage services, including Google Drive, Dropbox, and Amazon S3.

To get started with Rclone on your Mac, you'll need to install it first. You can download the Rclone installer from the official Rclone website.

Rclone can be configured to mount cloud storage as a local drive on your Mac, making it easy to access your files. This is done by creating a remote configuration file.

The remote configuration file contains the details of your cloud storage account, such as the service provider and your account credentials.

To mount Rclone on MacOS, you can use the built-in NFS server, macFUSE (also known as osxfuse), or FUSE-T.

You can also mount Rclone as a network drive on MacOS using the rclone command. To do this, create a new folder and edit a text file with the rclone command to mount the drive. The command should include the location of the rclone binary, the name of your Rclone remote, and the location of the folder where the network drive will be mounted.

Here are the steps to mount Rclone as a network drive on MacOS:

Note that you can also use the --nfs-cache-handle-limit option to control the maximum number of cached file handles stored by the nfsmount caching handler.

To mount Koofr as a network drive on macOS, you'll need to create a new folder or choose an existing one where you want your network drive to be located. Create a new folder located at /Users/koofr/KoofrDrive/ as an example.

You have two options to mount Koofr as a network drive. Option 1 is to automatically mount it at login, which requires editing a command and saving it to a specific path.

To automatically mount Koofr at login, you'll need to edit the command: "/usr/local/bin/rclone": /usr/local/bin/rclone is the location where you saved rclone on your computer. Replace it with the actual location where you saved rclone.

The command should be edited to include the following: "KoofrRemote:": KoofrRemote is the name of your remote. Replace it with the name of your remote, followed by a colon symbol (:). "/Users/Koofr/KoofrDrive/": /Users/Koofr/KoofrDrive/ is the location of the folder where the network drive is located. Replace it with the directory where you want to mount Koofr on your macOS system.

Save the edited command to the path: /Users/koofr/Library/LaunchAgents/net.koofr.rclone.plist. Replace /Users/koofr/ with a path to your home directory.

Alternatively, you can mount Koofr as a network drive manually by opening the Terminal and typing the command: rclone mount KoofrRemote: /Users/koofr/KoofrDrive/ --vfs-cache-mode full. Replace KoofrRemote with the name of your remote and /Users/koofr/KoofrDrive/ with the location of the folder you've created or chosen for your network drive.

Here are the steps to mount Koofr as a network drive manually:

1. Open the Terminal and type rclone mount KoofrRemote: /Users/koofr/KoofrDrive/ --vfs-cache-mode full

2. Replace KoofrRemote with the name of your remote

3. Replace /Users/koofr/KoofrDrive/ with the location of the folder you've created or chosen for your network drive

4. Press Enter to mount Koofr as a network drive

Note that the network drive will stay active only while the Terminal is open. Once you exit the Terminal, the network drive will no longer be visible on your computer.

On macOS, you might notice that file systems are case-insensitive, which means that two files with the same name but different cases are treated as the same file.

This can be a problem when syncing files from a source with case-sensitive file systems, like Linux. To fix this, you can use the --fix-case flag in rclone. This flag will rename files with incorrect casing to match the source file.

Here are some important things to know about the --fix-case flag:

By using the --fix-case flag, you can ensure that your files are synced correctly, even on a case-insensitive file system like macOS.

You can add custom HTTP headers to your Rclone mount on MacOS using the --header flag. This flag can be repeated to add multiple headers.

To add headers for all transactions, use the --header flag. It's supported for all HTTP based backends, making it a useful workaround for those not supported by --header-upload and --header-download.

You can also use --header-upload to add headers only for upload transactions. This flag can be repeated to add multiple headers.

If you need to add headers only for download transactions, use the --header-download flag. It can also be repeated to add multiple headers.

Remember to use the --header flag with care, as it may have unintended consequences on your Rclone mount.

Mounting Rclone on MacOS can be a bit tricky, but don't worry, I've got you covered.

Mounting with --read-only will fail silent attempts to write to files, unlike the clear warning you'd get with macFUSE.

You can mount Rclone with --read-only to ensure files aren't accidentally modified.

This can be particularly useful if you're working with critical data or need to preserve file integrity.

Mounting Rclone with --read-only is a simple way to prevent unwanted changes to your files.

The suffix option is a powerful tool in Rclone, allowing you to customize how files are backed up or transferred. The default suffix is .partial, but you can change it to any string up to 16 characters long.

If you're using the --inplace option, Rclone will use the --partial-suffix as a suffix for temporary files. This is useful for single file transfers, but if you want more control, look into --multi-thread-streams.

When using sync, copy, or move, any files that would be overwritten or deleted will have a suffix added to them. This is done using the --suffix option, which adds the specified suffix to the file name.

The remote in use must support server-side move or copy, and you must use the same remote as the destination of the sync. This is for use with files in the current directory or with --backup-dir.

If you're using --suffix without --backup-dir, it's recommended to put a filter rule in excluding the suffix, otherwise the sync will delete the backup files.

Using --suffix-keep-extension causes Rclone to put the suffix before the extension of backed up files. This can be helpful to ensure suffixed files can still be opened.

To get started with rclone on macOS, you'll need to have it installed on your local machine as well. This is necessary to generate an authentication token.

First, download rclone and macFuse. Once you have them downloaded, it's time to set up rclone. Open Terminal and type the following command: `rclone --config /Users/username/.rclone.conf config` to save rclone in the bin folder on your computer.

To configure rclone on macOS, you'll need to open a Terminal window and type `rclone authorize "box"`. This will prompt you to click on the "Use Single Sign On (SSO)" button on the web page that appears, followed by entering your USDA email address and completing the eAuthentication process.

The easiest way to make the config is to run rclone with the config option, which allows you to define as many storage paths as you like in the config file. You can use the `--config` entry to find the config file and choose its location.

Here's a list of the various storage systems that rclone supports, including Box, Dropbox, Google Cloud Storage, and many others:

Read Only mounts allow you to mount a drive with permissions that prevent writing to files. This is different from other systems, which may give a clear warning when you try to write to a file.

If you try to write to a file on a Read Only mount, the operation will fail silently. This is a change from how other systems, like macFUSE, handle this situation.

Read Only mounts can be useful for situations where you need to prevent accidental changes to important data.

To configure rclone, you'll need to create a config file. This file stores authentication information for various object storage systems, which can be complicated to set up.

You can create the config file by running rclone with the config option, which will guide you through the process. This is a good option if you're new to rclone and want to avoid accidentally deleting data.

The easiest way to make the config is to run rclone with the config option, which lists the available storage systems, including 1Fichier, Akamai Netstorage, Alias, Amazon S3, and many others.

Rclone syncs a directory tree from one storage system to another, and its syntax is like this: you can define as many storage paths as you like in the config file.

You can also set defaults for values in the config file on an individual remote basis. To do this, you'll need to set environment variables, which are documented in the page for each backend.

The names of the config items are documented in the page for each backend, and you can find the name of the environment variable by taking RCLONE_CONFIG_ + name of remote + _ + name of config file option and making it all uppercase.

Here's a list of some of the storage systems available in rclone:

You can also use environment variables to create a remote, which can be convenient if you're working with multiple storage systems.

Connection Strings are a powerful tool in Rclone, allowing you to modify existing remotes or create new ones on the fly. They can be used to add parameters to a remote, such as --drive-shared-with-me.

One major advantage of using Connection Strings is that they only apply to the remote in question, not to all remotes of that type. This is in contrast to using flags, which can affect both the source and destination remotes.

To use Connection Strings, you need to place the parameter in quotes, and if the parameter has a : or , you must place it in double quotes. For example, "shared_with_me=true". If a quoted value needs to include a quote, it should be doubled.

If you leave off the =parameter, Rclone will substitute it with =true, which works well with flags. For example, "s3=true" is equivalent to "s3=true" with the parameter explicitly set.

When using Connection Strings on the command line, you might need to surround them with " or ' to prevent the shell from interpreting special characters. On Linux/macOS, some characters are still interpreted inside " strings, so you can swap the roles of " and ' to avoid issues.

Connection Strings can also be used to add a suffix to the name of the remote based on the hash of the config, making it easier to tell modified remotes apart from unmodified ones when caching backends.

Valid remote names can contain a wide range of characters, including numbers, letters, underscores, hyphens, periods, plus signs, at symbols, and spaces.

However, there are some rules to keep in mind. For instance, remote names must not start with a hyphen or a space.

Remote names also must not end with a space. This can be a bit tricky, but it's essential to get it right.

If you're using a version of rclone older than 1.61, you'll need to stick to plain ASCII characters, which are limited to numbers 0-9, letters A-Z and a-z.

On the other hand, if you're using rclone version 1.61 or later, you can use any Unicode numbers and letters.

However, be cautious when using non-ASCII characters, especially on Windows, as the console traditionally uses a non-Unicode character set.

To avoid ambiguity, it's best to avoid using single character names on Windows, as they can be mistaken for Windows drive names.

If your remote names have spaces, you'll need to enclose them in double quotes, for example, "My Remote Name".

Rclone implements a metadata framework which can read metadata from an object and write it to the object when (and only when) it is being uploaded. This metadata is stored as a dictionary with string keys and string values.

The names of the keys have some limits. They must be lower case, may be a-z0-9 containing .- or _, and the length is backend dependent. Some backends can provide system metadata that it understands, while others can store arbitrary user metadata.

Where possible, the key names are standardized, so you can copy object metadata from one backend to another and the metadata will be translated appropriately.

Some backends have limits on the size of the metadata, and Rclone will give errors on upload if they are exceeded. This means you need to be mindful of the metadata size when working with certain backends.

When using sync, copy or move, any files which would have been overwritten or deleted are moved in their original hierarchy into the backup directory.

The backup directory must not overlap the destination directory without it being excluded by a filter rule. You can use today's date as the directory name passed to --backup-dir to store the old files.

The remote in use must support server-side move or copy and you must use the same remote as the destination of the sync. If running rclone from a script, you might want to pass --suffix with today's date.

On a very small number of remotes (FTP, SFTP, Dropbox for business), paths without a leading / will refer to your "home" directory and paths with a leading / will refer to the root, so be careful with your directory paths.

On most backends, remote:/path/to/dir refers to the same directory as remote:path/to/dir, so prefer that format.

Copy Dest Dir is a useful feature in rclone that allows you to check a directory in addition to the destination for files when using sync, copy, or move. This is especially helpful when you want to copy only files that have changed since the last backup.

The compare directory must not overlap the destination directory. You must use the same remote as the destination of the sync.

There are two options to use: --compare-dest and --copy-dest. --compare-dest is useful to copy just files that have changed since the last backup, while --copy-dest is useful for incremental backup.

Here's a summary of the differences between --compare-dest and --copy-dest:

Remember to use the same remote for the destination and the compare directory, and make sure the compare directory does not overlap with the destination directory.

You can customize the output of Rclone commands to display values in a more readable format. Rclone commands output values for sizes and counts either as raw numbers or in human-readable format by default.

The --human-readable option makes list commands output values in human-readable format instead of raw numbers. This option is useful for getting a better understanding of large numbers.

Rclone consistently uses binary units (powers of 2) for sizes and decimal units (powers of 10) for counts. The unit prefix for size is according to IEC standard notation, e.g. Ki for kibi.

Here's a breakdown of how Rclone displays human-readable values:

The about command outputs human-readable by default, with a command-specific option --full to output the raw numbers instead.

Ignoring files during a sync or transfer can be useful in certain situations. One such option is --ignore-existing, which makes rclone unconditionally skip all files that exist on the destination, no matter their content.

This option is particularly useful for cases where files change due to encryption, and it can be used with commands like move or moveto. However, it's worth noting that this option should be used with caution and is not generally recommended.

There's also --ignore-size, which causes rclone to skip verifying the sizes of files after transfer. This can be useful for transferring files to and from services like OneDrive, which occasionally misreport the size of image files.

By using these options, you can customize your rclone configuration to suit your specific needs and avoid unnecessary errors or complications.

The inplace flag is a powerful tool in rclone, allowing you to upload files directly to their final name without creating a temporary file.

This can be done by supplying the --inplace flag when uploading files to backends such as local, ftp, sftp, and pcloud. These backends have the PartialUploads feature flag set.

Without the --inplace flag, rclone will first upload to a temporary file with a hash of the source file's fingerprint and a .partial extension, and then rename it to the correct name once the upload is complete.

If the upload fails, the .partial file will be deleted, preventing partially uploaded files from being visible in directory listings and preventing overwriting of the old file.

Here are the backends that support inplace uploads:

Note that using inplace uploads can cause data loss of the existing file if the transfer fails, and it can also break hard links on the local file system.

Using the --no-traverse flag can save time when copying or moving files, especially if you're dealing with a large number of files on the destination.

It's not compatible with the sync command and will be ignored if used with it.

If you're only copying a small number of files or filtering most of them, --no-traverse can stop Rclone from listing the destination and save time.

However, if you're copying a large number of files that haven't changed, using --no-traverse can actually hinder the process.

Unicode Normalization is a crucial aspect of file syncing, especially when dealing with unicode characters in filenames.

Some operating systems, like macOS, store filenames in their decomposed form, which can lead to duplicate files if the data is copied back to a local filesystem.

Using the --no-unicode-normalization flag can disable this functionality, treating each unicode character as unique.

For example, é and é will be treated as unique characters, rather than being normalized into the same character.

The "-update" flag in rclone is a powerful tool for syncing files between different locations. It's used to force rclone to skip any files which exist on the destination and have a modified time that is newer than the source file.

This flag is particularly useful when transferring to a remote which doesn't support modification times directly, or when using --use-server-modtime to avoid extra API calls. It's more accurate than a --size-only check and faster than using --checksum.

You can use the "-u, --update" flag to force rclone to update files that have a modification time older than the source file's, if the sizes are different. If the sizes are the same, it will update the file if the checksum is different or not available.

Here's a summary of how the "-update" flag works:

This flag can be used in combination with other flags, such as --checksum and --modify-window, to get the desired behavior.

Server-side operations can work across different configurations, allowing you to perform tasks like copying or moving files between remotes that use the same backend but are configured differently.

This feature isn't enabled by default because it can be tricky for rclone to determine if it will work between any two configurations.

Some remotes support server-side copy, which can save time and resources by instructing the server to copy files in place instead of downloading and re-uploading them.

If you want to copy one folder to another, rclone will use server-side copy if the remote supports it, making the process much faster.

Server-side copies are identified in the log when using the -v flag, and they can be used to make aged backups efficiently by scripting.

Rclone will use the operating system's temp directory by default, but you can override this with the --temp-dir option. This option specifies the directory Rclone will use for temporary files.

On Unix systems, Rclone will use the value of the $TMPDIR environment variable if it's non-empty, otherwise it will default to /tmp.

On Windows, Rclone will use the first non-empty value from %TMP%, %TEMP%, %USERPROFILE%, or the Windows directory.

If you override the default with the --temp-dir option, the specified path will be set as the value of the TMPDIR environment variable on Unix systems, and the TMP and TEMP environment variables on Windows.

You can change the way rclone matches files for renaming by using the --track-renames-strategy option.

This option allows you to specify a combination of tokens that control the matching criteria, including modtime, hash, leaf, and size.

The default option is hash, but you can choose a different strategy by specifying a comma-separated list of tokens, such as modtime,leaf.

Using --track-renames-strategy modtime,leaf would match files based on modification time, the leaf of the file name, and the size.

Using --track-renames-strategy modtime or leaf can enable --track-renames support for encrypted destinations, which is not possible with the default hash strategy.

Here's a list of the available tokens:

Remember to choose a strategy that's compatible with your setup, especially if you're using an encrypted destination.

The ca-cert string array is a crucial configuration option for rclone. It loads PEM encoded certificate authority certificates to verify server certificates.

You'll need this flag if you've generated certificates signed with a local CA to connect to servers using those certificates.

Case sensitivity is a crucial aspect of remote names in rclone. Remote names are case sensitive, meaning that "Hello" and "hello" are considered two different names.

If you're using rclone on Windows, be cautious when using single character names, as they can be ambiguous with Windows drives' names. For example, a remote called "C" is indistinguishable from the C drive.

Rclone will always assume that a single letter name refers to a drive, so it's best to avoid using single character names on Windows. If your names have spaces, you'll need to put them in quotes, e.g. "My Remote".

The --fix-case flag can be used to fix case differences between source and destination filenames. This flag is ignored if --immutable is set, and it's not advisable to use --local-case-sensitive instead, as it can cause files to be deleted.

Here's a summary of the rules for remote names:

Note that these rules apply to both plain ASCII and Unicode characters, but if you're using older versions of rclone, you may need to be cautious when using characters that are not supported by all character encodings.

The directory cache is an essential aspect of rclone's configuration that can greatly impact its performance and accuracy.

You can control how long a directory should be considered up to date and not refreshed from the backend with the --dir-cache-time flag.

Changes made through the VFS will appear immediately or invalidate the cache, but changes made directly on the cloud storage will only be picked up once the directory cache expires if the backend doesn't support polling for changes.

If the backend supports polling, changes will be picked up within the polling interval.

You can send a SIGHUP signal to rclone to flush all directory caches, regardless of how old they are.

Assuming only one rclone instance is running, you can reset the cache like this:

Fingerprinting is a process used by the VFS to check if a local file copy has changed relative to a remote file. It's made from three attributes: size, modification time, and hash, where available on an object.

These attributes are used to create a unique fingerprint for each file. However, some backends can be slow to read these attributes, requiring extra API calls or work per object.

For example, the local and sftp backends are slow to read the hash attribute because they have to read the entire file and hash it. Similarly, the s3, swift, ftp, and qinqstor backends are slow to read the modification time attribute because they need to do an extra API call to fetch it.

Fortunately, there's a way to speed up fingerprinting without sacrificing too much accuracy. By using the --vfs-fast-fingerprint flag, rclone will exclude slow operations from the fingerprint, making it faster but less accurate.

This flag is recommended for users running a vfs cache over local, s3, or swift backends. If you change the value of this flag, the fingerprints of the files in the cache may be invalidated, and the files will need to be downloaded again.