RClone - Google Drive

From Tech Wiki
Revision as of 14:33, 30 April 2016 by Adam.birds (talk | contribs) (Created page with "=Google Drive= ==Setup== Paths are specified as '''drive:path''' Drive paths may be as deep as required, eg '''drive:directory/subdirectory'''. The initial setup for drive...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Google Drive

Setup

Paths are specified as drive:path

Drive paths may be as deep as required, eg drive:directory/subdirectory.

The initial setup for drive involves getting a token from Google drive which you need to do in your browser. rclone config walks you through it.

Here is an example of how to make a remote called remote. First run:

rclone config

This will guide you through an interactive setup process:

n) New remote
d) Delete remote
q) Quit config
e/n/d/q> n
name> remote
Type of storage to configure.
Choose a number from below, or type in your own value
 1 / Amazon Cloud Drive
   \ "amazon cloud drive"
 2 / Amazon S3 (also Dreamhost, Ceph)
   \ "s3"
 3 / Backblaze B2
   \ "b2"
 4 / Dropbox
   \ "dropbox"
 5 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
 6 / Google Drive
   \ "drive"
 7 / Hubic
   \ "hubic"
 8 / Local Disk
   \ "local"
 9 / Microsoft OneDrive
   \ "onedrive"
10 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
11 / Yandex Disk
   \ "yandex"
Storage> 6
Google Application Client Id - leave blank normally.
client_id> 
Google Application Client Secret - leave blank normally.
client_secret> 
Remote config
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine or Y didn't work
y) Yes
n) No
y/n> y
If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
Log in and authorize rclone for access
Waiting for code...
Got code
--------------------
[remote]
client_id = 
client_secret = 
token = {"AccessToken":"xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx","Expiry":"2014-03-16T13:57:58.955387075Z","Extra":null}
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y

Note that rclone runs a webserver on your local machine to collect the token as returned from Google if you use auto config mode. This only runs from the moment it opens your browser to the moment you get back the verification code. This is on http://127.0.0.1:53682/ and this it may require you to unblock it temporarily if you are running a host firewall, or use manual mode.

Usage

You can then use it like this,

List directories in top level of your drive:

rclone lsd remote:

List all the files in your drive:

rclone ls remote:

To copy a local directory to a drive directory called backup:

rclone copy /home/source remote:backup

Modified Time

Google drive stores modification times accurate to 1 ms.

Revisions

Google drive stores revisions of files. When you upload a change to an existing file to google drive using rclone it will create a new revision of that file.

Revisions follow the standard google policy which at time of writing was:

  • They are deleted after 30 days or 100 revisions (whatever comes first).
  • They do not count towards a user storage quota.

Deleting Files

By default rclone will delete files permanently when requested. If sending them to the trash is required instead then use the --drive-use-trash flag.

Specific Options

–drive-chunk-size=SIZE

Upload chunk size. Must a power of 2 >= 256k. Default value is 8 MB.

Making this larger will improve performance, but note that each chunk is buffered in memory one per transfer.

Reducing this will reduce memory usage but decrease performance.

–drive-full-list

No longer does anything - kept for backwards compatibility.

–drive-upload-cutoff=SIZE

File size cutoff for switching to chunked upload. Default is 8 MB.

–drive-use-trash

Send files to the trash instead of deleting permanently. Defaults to off, namely deleting files permanently.

–drive-auth-owner-only

Only consider files owned by the authenticated user. Requires that –drive-full-list=true (default).

–drive-formats

Google documents can only be exported from Google drive. When rclone downloads a Google doc it chooses a format to download depending upon this setting.

By default the formats are docx,xlsx,pptx,svg which are a sensible default for an editable document.

When choosing a format, rclone runs down the list provided in order and chooses the first file format the doc can be exported as from the list. If the file can’t be exported to a format on the formats list, then rclone will choose a format from the default list.

If you prefer an archive copy then you might use --drive-formats pdf, or if you prefer openoffice/libreoffice formats you might use --drive-formats ods,odt.

Note that rclone adds the extension to the google doc, so if it is calles My Spreadsheet on google docs, it will be exported as My Spreadsheet.xlsx or My Spreadsheet.pdf etc.

Here are the possible extensions with their corresponding mime types:

Extension Mime Type Description
csv text/csv Standard CSV format for Spreadsheets
doc application/msword Micosoft Office Document
docx application/vnd.openxmlformats-officedocument.wordprocessingml.document Microsoft Office Document
html text/html An HTML Document
jpg image/jpeg A JPEG Image File
ods application/vnd.oasis.opendocument.spreadsheet Openoffice Spreadsheet
odt application/vnd.oasis.opendocument.text Openoffice Document
pdf application/pdf Adobe PDF Format
png image/png PNG Image Format
pptx application/vnd.openxmlformats-officedocument.presentationml.presentation Microsoft Office Powerpoint
rtf application/rtf Rich Text Format
svg image/svg+xml Scalable Vector Graphics Format
txt text/plain Plain Text
xls application/vnd.ms-excel Microsoft Office Spreadsheet
xlsx application/vnd.openxmlformats-officedocument.spreadsheetml.sheet Microsoft Office Spreadsheet
zip application/zip A ZIP file of HTML, Images CSS

Limitations

Drive has quite a lot of rate limiting. This causes rclone to be limited to transferring about 2 files per second only. Individual files may be transferred much faster at 100s of MBytes/s but lots of small files can take a long time.