Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: pantheon rsync

Status
colourYellow
titleDRAFT

...

Using Drush's built-in RSYNC

note that drush rsync can be funky, especially with older versions of drush. Real rsync in generally recommended.

Basic Usage

Drush aliases makes operations destinations clear. 

Here is some basic usage. We pull images down from dev to our local. Drupal will find the path to each instance's %files directory. Magic.

Note that the slash appears slash appears on the source, but not the target destination.

...

Code Block
themeFadeToGrey
languagebash
$ drush rsync --exclude=private @mysite.live:%files/ @mysite.local:%files
Info

note that you cannot use drush to sync 2 remote instances, like live to test. That would be an on-server operation.

Safety first: passing flags

...

Code Block
themeFadeToGrey
languagebash
$ drush rsync -y --mode=razog --exclude-paths=%files/private:%files/css:%files/js:%files/styles:%files/ctools:%files/backup_migrate @mysite.local:%files/ @mysite.dev:%files

 

Using native rsync for full control of file

...

transfers

 

Here is a verbose exclude list using straight rsync:

...

Code Block
themeFadeToGrey
languagebash
$ ssh mysite.test

 

  -e 'ssh -axp 1022' 

 

By default, symbolic links are not transferred at all.

...

title
Code Block
themeFadeToGrey
languagebash
css
civicrm
ctools
js
googleanalytics
xmlsitemap
backup_migrate
tmp
.DS_Store
.DS_Store*
.sass-cache
ehthumbs.db
Icon?
Thumbs.db
._*
*.bak
*_cache
*.tgz
*.un~
.buildpath
.project
.settings
*.sublime-project
*.sublime-workspace
*.sublime-projectcompletions
nbproject
Code Block

 

rsync

...

to pantheon 

Grab the pantheon credentials from the desitnation account and modify to fit.

Log in to the client server to run this operation, though you can sync a local directory in just the same way:

Code Block
languagebash
# rsync the files to Panthon dev
rsync -trazog --verbose --exclude-from='/var/www/backup/rsync-files-exclude.txt' \
  /var/www/mysite/htdocs/sites/default/files/ \
  --ipv4 -e 'ssh -p 2222' \
  dev.b37d48af-d7ab-4d1e-b6cf-XXXXXXXXXXX@appserver.dev.b37d48af-d7ab-4d1e-b6cf-XXXXXXXXXXX.drush.in:files;
# enter pantheon dashboard password when prompted

 

...

Resources

 

 

Code Block
titlersync options
collapsetrue
 -v, --verbose               increase verbosity
 -q, --quiet                skip basedsuppress on checksum, not mod-time & sizenon-error messages
     -a, -no-archivemotd               archivesuppress daemon-mode; MOTD equals -rlptgoD (no(see caveat)
 -Hc, -A,-X)checksum      --no-OPTION        skip based on checksum, not turnmod-time off& ansize
implied OPTION (e.g. -a, --no-D)archive  -r, --recursive            archive recursemode; intoequals directories-rlptgoD (no -RH, -A,-relativeX)
     --no-OPTION        use relative path names  turn off an  --no-implied-dirs OPTION      don't send implied dirs with --relative(e.g. --no-D)
 -br, --backuprecursive             recurse into  makedirectories
backups (see --suffix &-R, --backup-dir)relative      --backup-dir=DIR        makeuse backupsrelative into hierarchy based in DIRpath names
     --suffix=SUFFIXno-implied-dirs       don't send backupimplied suffixdirs (default ~ w/o --backup-dir)with --relative
 -ub, --updatebackup                skipmake filesbackups that are newer on the receiver(see --suffix & --backup-dir)
       --inplacebackup-dir=DIR        make backups into hierarchy based in DIR
update destination files in-place  --suffix=SUFFIX    --append     backup suffix (default ~ w/o --backup-dir)
 -u, --update   append data onto shorter files      --append-verify   skip files that are newer on --append w/old data in file checksumthe receiver
     -d, --dirsinplace               update destination files transferin-place
directories without recursing  -l, --linksappend                append data onto shorter copyfiles
symlinks as symlinks  -L, --copyappend-linksverify         --append w/old data transformin symlinkfile intochecksum
referent file/dir
 -d, --dirs     --copy-unsafe-links     only "unsafe" symlinks are transformed    transfer directories without recursing
 -l, -safe-links              ignore symlinks that pointcopy outsidesymlinks theas treesymlinks
 -kL, --copy-dirlinkslinks            transform symlink tointo referent file/dir
 into referent dir  -K, -copy-keepunsafe-dirlinkslinks     only "unsafe" symlinks are treattransformed
symlinked dir on receiver as dir
 -H, --hardsafe-links            preserve hard linksignore symlinks that point outside the tree
 -pk, --permscopy-dirlinks         transform symlink to dir into referent   preserve permissionsdir
 -EK, --executabilitykeep-dirlinks         treat symlinked preservedir executabilityon receiver as dir
 -H, --chmod=CHMODhard-links           affect filepreserve and/or directory permissionshard links
 -Ap, --aclsperms                  preserve ACLspermissions
(implies -p)
 -XE, --xattrsexecutability         preserve    executability
  preserve extended attributes  -o, --owner-chmod=CHMOD           affect file and/or directory permissions
 preserve owner (super-user only)
 -g-A, --groupacls                  preserve groupACLs (implies -p)
  -X, --devicesxattrs                preserve deviceextended filesattributes
 (super-user only)o, --owner      --specials              preserve special files
 -D    owner (super-user only)
 -g, --group                 preserve group
   same as --devices --specials  -t, --times           preserve device files (super-user only)
 preserve modification times  -O, --omit-dir-times-specials        omit  directories from --times  preserve special files
 --superD                      receiver attempts super-user activities same as --devices --specials
 -t, -fake-supertimes               store/recover privileged attrspreserve usingmodification xattrstimes
 -SO, --sparseomit-dir-times        omit directories from --times
    handle sparse--super files efficiently  -n, --dry-run             receiver attempts performsuper-user aactivities
trial run with no changes made  -W, --wholefake-filesuper            copystore/recover filesprivileged whole (w/o delta-xfer algorithm)attrs using xattrs
 -xS, --one-file-systemsparse       don't cross filesystem boundaries  -B, --block-size=SIZE   handle sparse files efficiently
force a fixed checksum block-size
 -e, --rsh=COMMAND-n, --dry-run              specify theperform remotea shelltrial torun usewith no changes made
 -W, --rsyncwhole-path=PROGRAMfile    specify the rsync to run on remote machine copy files    --existingwhole (w/o delta-xfer algorithm)
 -x, --one-file-system       don't cross filesystem boundaries
 -B,  skip creating new files on receiver
     --ignore-existing--block-size=SIZE       force a fixed checksum block-size
 -e, --rsh=COMMAND       skip updating files that existspecify onthe receiverremote shell to use
     --remove-source-filesrsync-path=PROGRAM   sender removesspecify synchronizedthe files (non-dir)
     --delrsync to run on remote machine
     --existing              skip ancreating new aliasfiles for --delete-duringon receiver
     --deleteignore-existing       skip updating files that exist on receiver
   delete extraneous --remove-source-files from dest dirs sender removes synchronized files (non-dir)
     --delete-beforedel            receiver deletes before transfer (default)   an alias for --delete-during
     --delete   receiver deletes during xfer, not before      --delete-delay  delete extraneous files from dest dirs
  find deletions during, delete after
     --delete-after before         receiver deletes afterbefore transfer, not(default)
before      --delete-excludedduring        also deletereceiver excludeddeletes filesduring fromxfer, destnot dirsbefore
     --ignoredelete-errorsdelay         delete evenfind ifdeletions thereduring, aredelete I/Oafter
errors      --delete-forceafter          receiver deletes after transfer, not before
 force deletion of dirs even--delete-excluded if not empty    also  --max-delete=NUM        don't delete more than NUM filesdelete excluded files from dest dirs
     --maxignore-size=SIZEerrors         don'tdelete transfereven anyif filethere largerare thanI/O SIZEerrors
     --min-size=SIZEforce         don't transfer any file smaller than SIZE  force deletion of  --partialdirs even if not empty
     --max-delete=NUM        don't delete keepmore partiallythan transferredNUM files
     --partialmax-dirsize=DIRSIZE        put adon't partiallytransfer transferredany file larger intothan DIRSIZE
     --delaymin-updatessize=SIZE         putdon't alltransfer updatedany filesfile intosmaller placethan atSIZE
end    -m, --prune-empty-dirspartial      prune empty directory chains from file-list    keep partially --numeric-idstransferred files
     --partial-dir=DIR    don't map uid/gid values by user/group name   put a partially transferred file into DIR
     --timeout=SECONDSdelay-updates       set I/O timeout in seconds
     --contimeout=SECONDSput all updated files into place at end
 -m, --prune-empty-dirs    set daemon connectionprune timeoutempty indirectory secondschains from -I,file-list
     --ignorenumeric-timesids           don't skipmap filesuid/gid thatvalues matchby sizeuser/group andname
time
     --size-onlytimeout=SECONDS       set      skip files that match in sizeI/O timeout in seconds
     --modify-windowcontimeout=NUMSECONDS    set comparedaemon mod-timesconnection withtimeout reducedin accuracyseconds
 -TI, --tempignore-dir=DIRtimes          createdon't temporaryskip files inthat directorymatch DIRsize and -y, --fuzzytime
     --size-only            find similarskip filefiles forthat basismatch ifin nosize
dest file      --comparemodify-destwindow=DIRNUM      also compare receivedmod-times fileswith relativereduced toaccuracy
DIR      -T, --copytemp-destdir=DIR         ... andcreate includetemporary copiesfiles ofin unchangeddirectory filesDIR
 -y, --fuzzy   --link-dest=DIR           hardlink to files infind DIRsimilar whenfile unchangedfor basis -z, --compress  if no dest file
     --compare-dest=DIR      compressalso compare filereceived datafiles duringrelative theto transferDIR
     --compresscopy-leveldest=NUMDIR    explicitly set compression level  ... and include copies  --skip-compress=LIST    skip compressingof unchanged files
with suffix in LIST
 -C, --cvslink-excludedest=DIR         hardlink to auto-ignore files in theDIR samewhen wayunchanged
CVS does
 -fz, --filter=RULEcompress           add a file-filtering RULEcompress file -Fdata during the transfer
     --compress-level=NUM    explicitly set compression level
         same as --filter='dir-merge /.rsync-filter'--skip-compress=LIST    skip compressing files with suffix in LIST
 -C, --cvs-exclude           auto-ignore files in the same way CVS does
 repeated:-f, --filter='- .rsync-filter'RULE       --exclude=PATTERN    add a file-filtering excludeRULE
files matching-F PATTERN      --exclude-from=FILE     read exclude patterns from FILE      --include=PATTERN    same as  don't exclude files matching PATTERN--filter='dir-merge /.rsync-filter'
         --include-from=FILE     read include patterns from FILE      --files-from=FILE     repeated:  read list of source-file names from FILE
 -0, --from0    --filter='- .rsync-filter'
     --exclude=PATTERN       exclude files matching PATTERN
     --exclude-from=FILE     read exclude allpatterns *from/filter filesFILE
are delimited by 0s  -s, --protect-args-include=PATTERN       don't exclude files no space-splitting; wildcard chars onlymatching PATTERN
        --include-addressfrom=ADDRESSFILE     read include bindpatterns addressfrom for outgoingFILE
socket to daemon      --files-portfrom=PORTFILE       read list of source-file names from FILE
specify double-colon alternate port number0, --from0        --sockopts=OPTIONS      specify custom TCP options all *from/filter files are delimited by 0s
 -s, --blockingprotect-ioargs          no use blocking I/O for the remote shellspace-splitting; wildcard chars only
        --statsaddress=ADDRESS       bind address for outgoing socket to daemon
   give some file--transferport=PORT stats  -8, --8-bit-output         specify leave high-bit chars unescaped in output
 -h, --human-readabledouble-colon alternate port number
     --sockopts=OPTIONS      specify custom outputTCP numbersoptions
in a human-readable format      --progressblocking-io           use blocking I/O showfor progressthe duringremote transfershell
 -P    --stats                 give some file-transfer stats
 same-8, as --partial 8-bit-progressoutput  -i, --itemize-changes       outputleave a changehigh-summarybit forchars allunescaped updatesin output
 -h,   --out-format=FORMAThuman-readable        output updatesnumbers usingin thea specifiedhuman-readable FORMATformat
     --log-file=FILEprogress         log what we're doing to theshow specifiedprogress FILEduring    transfer
 --log-file-format=FMTP   log updates using the specified FMT      --password-file=FILE    read daemon-access password from FILE    same as --listpartial --onlyprogress
 -i, --itemize-changes       output a change-summary listfor theall filesupdates
instead of copying them      --out-bwlimitformat=KBPSFORMAT     output updates using the  limit I/O bandwidth; KBytes per secondspecified FORMAT
     --writelog-batchfile=FILE        write log awhat batchedwe're updatedoing to the specified FILE
     --onlylog-writefile-batchformat=FILEFMT like --write-batch butlog w/o updating destupdates using the specified FMT
     --readpassword-batchfile=FILE       read adaemon-access batchedpassword update from FILE
     --protocol=NUMlist-only            force anlist olderthe protocolfiles versioninstead toof becopying usedthem
     --iconvbwlimit=CONVERT_SPECKBPS    request charset conversion of filenames  limit I/O bandwidth;  --checksum-seed=NUM     set block/file checksum seed (advanced)
 -4, --ipv4KBytes per second
     --write-batch=FILE      write a batched update to FILE
      prefer IPv4
 -6, --ipv6    --only-write-batch=FILE like --write-batch but w/o updating dest
     --read-batch=FILE       read prefera batched IPv6update from FILE
   --version  --protocol=NUM          force an older protocol version to be used
 print version number (-h) --helpiconv=CONVERT_SPEC    request charset conversion of filenames
     --checksum-seed=NUM     set showblock/file thischecksum helpseed (seeadvanced)
below for-4, -h comment)

 

Resources

 

...

-ipv4                  prefer IPv4
 -6, --ipv6                  prefer IPv6
     --version               print version number
(-h) --help                  show this help (see below for -h comment)