Status | ||||
---|---|---|---|---|
|
...
Using Drush's built-in RSYNC
Info |
---|
note that drush rsync can be funky, especially with older versions of drush. Real rsync in generally recommended. |
Basic Usage
Drush aliases makes operations destinations clear.
Here is some basic usage. We pull images down from dev to our local. Drupal will find the path to each instance's %files directory. Magic.
Note that the slash appears slash appears on the source, but not the target destination.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
$ drush rsync @mysite.dev:%files/images/ @mysite.local:%files/images |
Here, we pull down all files from live to local, but exclude drupal's private files
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
$ drush rsync --exclude=private @mysite.live:%files/ @mysite.local:%files |
Info |
---|
note that you cannot use drush to sync 2 remote instances, like live to test. That would be an on-server operation. |
Safety first: passing flags
...
Drush has some option flags of its own we can send to test operations before committing. In this case we are going from our local to staging:
Code Block | ||||
---|---|---|---|---|
| ||||
$ drush --simulate --debug rsync --mode=razoglp @sitename.local:%files/images/ @sitename.staging:%files/images |
Excluding Paths
Drush provides the ability to leverage parts of rsync, but will not easily exclude files based on patterns. We can exclude paths, however:
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
$ drush rsync -y --mode=razog --exclude-paths=%files/private:%files/css:%files/js:%files/styles:%files/ctools:%files/backup_migrate @mysite.localdev:%files/ @mysite.devlocal:%files/ |
Using native rsync for full control of file
...
transfers
Here is a verbose exclude list using straight rsync:
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
$ rsync -razv --delete --exclude=css --exclude=*_cache --exclude=js --exclude=googleanalytics --exclude=xmlsitemap \
--exclude=backup_migrate --exclude=styles user@domain.com:/path/to/remote/files/ \
/path/to/local/files/ |
We exclude a number of directories here and also delete anything on our local that does not exist on the remote.
setting hosts
With native rsync, we don't use drush aliases. However, we can set up shortcuts to hosts in .ssh/config.
Code Block | ||
---|---|---|
| ||
# mysite Host mysite.dev Hostname mysite.com port 2222 Compression yes User username Host mysite.test Hostname test.mysite.com port 2222 Compression yes User username |
...
These greatly simplify logging in via SSH. Now, instead of
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
$ ssh -p 2222 username@test.mysite.com |
you can simply do:
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
$ ssh mysite.test |
...
-e 'ssh -axp 1022'
By default, symbolic links are not transferred at all.
We can replace multiple exclude patterns in the rsync command with a reference to an external file, using a flag, like so: --exclude-from '/var/www/backup/rsync-exclude.txt'
Here are files and directories to exclude from drupal's files directory.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
css
civicrm
ctools
js
googleanalytics
xmlsitemap
backup_migrate
tmp
styles
.DS_Store
.DS_Store*
.sass-cache
ehthumbs.db
Icon?
Thumbs.db
._*
*.bak
*_cache
*.tgz
*.un~
.buildpath
.project
.settings
*.sublime-project
*.sublime-workspace
*.sublime-projectcompletions
nbproject | ||||||
Code Block | ||||||
rsync
...
to pantheon
Grab the pantheon credentials from the desitnation account and modify to fit.
Log in to the client server to run this operation, though you can sync a local directory to Pantheon in much the same way.
You will need to get the ssh key for the machine performing the rsync operation to Pantheon up to your control panel if it is not already there.
Code Block | ||
---|---|---|
| ||
# rsync the files to Panthon dev
rsync -trazog --verbose --exclude-from='/var/www/backup/rsync-files-exclude.txt' \
/var/www/mysite/htdocs/sites/default/files/ \
--ipv4 -e 'ssh -p 2222' \
dev.b37d48af-d7ab-4d1e-b6cf-XXXXXXXXXXX@appserver.dev.b37d48af-d7ab-4d1e-b6cf-XXXXXXXXXXX.drush.in:files;
# enter pantheon dashboard password when prompted |
...
Resources
Code Block | ||||
---|---|---|---|---|
| ||||
-v, --verbose turn off anincrease implied OPTION (e.g. --no-D) -rverbosity -q, --recursivequiet recurse into directories -R,suppress non--relativeerror messages --no-motd use relative path names suppress daemon--no-implied-dirsmode MOTD (see caveat) -c, --checksum don't send implied dirs with --relative skip based on checksum, not mod-time & size -ba, --backuparchive archive make backups (seemode; equals --suffixrlptgoD &(no -H,-backupA,-dirX) --backupno-dir=DIROPTION make backups into hierarchy based inturn DIRoff an implied --suffix=SUFFIX backup suffix (default ~ w/oOPTION (e.g. --backupno-dirD) -ur, --updaterecursive recurse into directories skip files that are newer on the receiver-R, --relative --inplace use relative path names --no-implied-dirs update destination files in-place don't send implied dirs with --appendrelative -b, --backup append data onto shortermake filesbackups (see --suffix & --appendbackup-verifydir) --backup-append w/old data in file checksum -d, --dirs dir=DIR make backups into hierarchy based in DIR --suffix=SUFFIX backup transfer directories without recursingsuffix (default ~ w/o --backup-dir) -lu, --linksupdate skip files that are newer on the copyreceiver symlinks as symlinks -L, --copy-linksinplace transform symlink into referentupdate file/dirdestination files in-place - -copy-unsafe-links-append only "unsafe" symlinks are transformedappend data onto shorter files --safe-links --append-verify ignore symlinks that point outside the tree--append w/old data in file checksum -kd, --copy-dirlinksdirs transform symlink to dir into referent dir -K, --keep-dirlinks transfer directories without recursing -l, --links treat symlinked dir oncopy receiversymlinks as dirsymlinks -HL, --hardcopy-links preservetransform hardsymlink linksinto -p, --permsreferent file/dir --copy-unsafe-links only "unsafe" symlinks are transformed preserve permissions -E, -safe-executabilitylinks preserve executability --chmod=CHMOD ignore symlinks that point outside the tree -k, --copy-dirlinks transform symlink affectto filedir and/orinto directoryreferent permissionsdir -AK, --aclskeep-dirlinks treat symlinked dir on receiver as dir preserve ACLs (implies -p) -X, --xattrs-H, --hard-links preserve hard links preserve extended attributes -o-p, --ownerperms preserve owner (super-user only) -gpermissions -E, --group executability preserve groupexecutability --deviceschmod=CHMOD affect file and/or directory preservepermissions device files (super-user only)-A, --acls --specials preserve ACLs (implies preserve-p) special files-X, --Dxattrs preserve same as --devices --specials -textended attributes -o, --timesowner preserve modification timesowner (super-user only) -Og, --omit-dir-timesgroup omit directories from --times preserve --supergroup --devices receiver attempts super preserve device files (super-user activitiesonly) --fake-superspecials store/recover privileged attrspreserve usingspecial xattrsfiles -S, --sparseD handle sparse files efficientlysame as --n,devices --dry-runspecials -t, --times perform a trial run with nopreserve changesmodification madetimes -WO, --whole-fileomit-dir-times omit directories from --times copy files whole (w/o delta-xfer algorithm) -x, --one-file-system--super don't cross filesystem boundariesreceiver attempts -B,super-user activities --block-size=SIZEfake-super force a fixed checksum block-size store/recover privileged attrs using xattrs -eS, --rsh=COMMANDsparse specify the remote shell to usehandle sparse files efficiently -n, --rsyncdry-path=PROGRAMrun specify the rsync to run on remote machine perform a --existingtrial run with no changes made -W, --whole-file skip creating new files on receiver copy files whole (w/o --ignore-existing delta-xfer algorithm) -x, --one-file-system skip updating files that existdon't oncross receiverfilesystem boundaries -B, --remove-source-filesblock-size=SIZE sender removes synchronized files (non-dir)force a fixed checksum block-size --dele, --rsh=COMMAND specify the remote shell to use an alias for --delete-duringrsync-path=PROGRAM specify the rsync to run on remote machine --deleteexisting skip creating delete extraneousnew files fromon dest dirsreceiver --deleteignore-beforeexisting skip updating receiverfiles deletesthat beforeexist transferon (default)receiver --deleteremove-source-duringfiles sender removes synchronized files (non-dir) receiver deletes during xfer, not--del before --delete-delay find deletions during,an deletealias afterfor --delete-afterduring --delete receiver deletes after transfer, not before --delete-excluded also delete excludedextraneous files from dest dirs --ignoredelete-errorsbefore deletereceiver evendeletes ifbefore there are I/O errorstransfer (default) --forcedelete-during receiver deletes during xfer, not before force deletion of--delete-delay dirs even if not empty --max-delete=NUM find deletions during, delete after don't --delete-after more than NUM files receiver deletes after transfer, not before --maxdelete-size=SIZEexcluded also delete don'texcluded transferfiles anyfrom filedest largerdirs than SIZE --minignore-size=SIZEerrors don'tdelete transfereven anyif filethere smallerare thanI/O SIZEerrors --partialforce keep partially transferred files force deletion of dirs even if not empty --partialmax-dirdelete=DIRNUM put adon't partiallydelete transferredmore filethan intoNUM DIRfiles --delay-updatesmax-size=SIZE don't transfer putany allfile updatedlarger filesthan intoSIZE place at end -m, --prune-empty-dirsmin-size=SIZE prune empty directorydon't chainstransfer fromany file-list smaller than SIZE --numeric-idspartial don't map uid/gid valueskeep bypartially user/grouptransferred namefiles --partial-timeoutdir=SECONDSDIR put set I/O timeout in secondsa partially transferred file into DIR --delay-contimeout=SECONDSupdates set daemon connection timeout input secondsall updated -I, --ignore-times files into place at end -m, --prune-empty-dirs don't skip filesprune thatempty matchdirectory sizechains andfrom timefile-list --sizenumeric-onlyids don't map skipuid/gid filesvalues thatby matchuser/group inname size --modify-windowtimeout=NUMSECONDS compare mod-times withset reducedI/O accuracytimeout in -T, --temp-dir=DIRseconds --contimeout=SECONDS set createdaemon temporaryconnection filestimeout in directoryseconds DIR -yI, --ignore-fuzzytimes don't skip files that match size and findtime similar file for basis if no--size-only dest file --compare-dest=DIR skip alsofiles comparethat receivedmatch filesin relative tosize DIR --copymodify-destwindow=DIRNUM compare mod-times with reduced ...accuracy and include copies of unchanged files --link-dest -T, --temp-dir=DIR hardlink tocreate temporary files in directory DIR when unchanged -zy, --compressfuzzy find compresssimilar file data during the transferfor basis if no dest file --compresscompare-level=NUM explicitly set compression leveldest=DIR also compare received files relative to DIR --skipcopy-compressdest=LISTDIR skip compressing files with suffix in... LISTand -C,include copies of unchanged files --cvslink-excludedest=DIR hardlink auto-ignoreto files in theDIR samewhen way CVSunchanged does -fz, --filter=RULEcompress add a file-filtering RULEcompress file -Fdata during the transfer --compress-level=NUM explicitly set compression level --skip-compress=LIST same as --filter='dir-merge /.rsync-filter' skip compressing files with suffix in LIST -C, --cvs-exclude auto-ignore files in the same way CVS does repeated: -f, --filter='- .rsync-filter'RULE --exclude=PATTERN add a exclude files matching PATTERNfile-filtering RULE -F --exclude-from=FILE read exclude patterns from FILE --include=PATTERN same as don't exclude files matching PATTERN--filter='dir-merge /.rsync-filter' --include-from=FILE read include patterns from FILE --files-from=FILE read list of source-file names from FILE -0, --from0repeated: --filter='- .rsync-filter' --exclude=PATTERN exclude files matching PATTERN all *from/filter files are delimited by 0s -s, --protect-args --exclude-from=FILE read exclude patterns from FILE --include=PATTERN no space-splitting; wildcard chars onlydon't exclude files matching PATTERN --address=ADDRESS --include-from=FILE bind address forread outgoinginclude socketpatterns tofrom daemonFILE --portfiles-from=PORTFILE read list specify double-colon alternate port number --sockopts=OPTIONSof source-file names from FILE -0, --from0 specify custom TCP options all *from/filter files are delimited by 0s -s, --blockingprotect-ioargs no use blocking I/O for the remote shellspace-splitting; wildcard chars only --statsaddress=ADDRESS bind address for outgoing socket to daemon give some file-transfer stats -8, --8-bit-output--port=PORT specify double-colon alternate leave high-bit chars unescaped in output -h, --human-readableport number --sockopts=OPTIONS specify custom outputTCP numbersoptions in a human-readable format --blocking-progressio use blocking I/O showfor progressthe duringremote transfershell -P --stats give same as --partial --progresssome file-transfer stats -i8, --8-itemizebit-changesoutput output a change-summary forleave allhigh-bit updateschars unescaped in output -h, --out-format=FORMAThuman-readable output updatesnumbers usingin thea specifiedhuman-readable FORMATformat --log-file=FILEprogress log what we're doing toshow theprogress specifiedduring FILEtransfer -P --log-file-format=FMT log updates using the specified FMT --password-file=FILE read daemon-access password from FILEsame as --partial --progress -i, --listitemize-onlychanges output a change-summary for all updates list the files instead of copying them --bwlimit=KBPS --out-format=FORMAT output updates using the limit I/O bandwidth; KBytes per secondspecified FORMAT --writelog-batchfile=FILE write a batched update to log what we're doing to the specified FILE --onlylog-writefile-batchformat=FILEFMT like --write-batch but w/o updating dest log updates using the specified FMT --readpassword-batchfile=FILE read a batched updatedaemon-access password from FILE --protocol=NUMlist-only force anlist olderthe protocolfiles versioninstead toof becopying usedthem --iconvbwlimit=CONVERT_SPECKBPS request charset conversion of filenames limit I/O bandwidth; --checksum-seed=NUM KBytes per second set block/file checksum seed (advanced) -4, -write-ipv4batch=FILE write a batched update to FILE prefer IPv4 -6, --ipv6 --only-write-batch=FILE like --write-batch but w/o updating dest --read-batch=FILE read prefer IPv6a batched update from FILE --versionprotocol=NUM force an older protocol version to be used print version number (-h) --helpiconv=CONVERT_SPEC request charset conversion of filenames --checksum-seed=NUM set showblock/file thischecksum helpseed (seeadvanced) below for-4, --h comment) |
Resources
...
ipv4 prefer IPv4
-6, --ipv6 prefer IPv6
--version print version number
(-h) --help show this help (see below for -h comment) |