Use Robocopy to copy only changed files?

Question

I'm trying to find an easy way of deploying only changed files to the webserver for deployment purpose. In times past I've used MSBuild, which could be told to only copy files that were newer than the ones on the target, but I'm in a hurry and don't want to try to figure out the newer version of MSBuild.

Can I use ROBOCOPY for this? There is a list of options for exclusion, which is:

• /XC :: eXclude Changed files.
• /XN :: eXclude Newer files.
• /XO :: eXclude Older files.
• /XX :: eXclude eXtra files and directories.
• /XL :: eXclude Lonely files and directories.

What exactly does it mean to exclude? Exclude copying, or exclude overwriting? For example, if I wrote:

ROBOCOPY C:\SourceFolder\ABC.dll D:\DestinationFolder /XO

would this copy only newer files, not files of the same age?

Or is there a better tool to do this?

Footlight

Huh? What's there to figure out about MSBuild? If you have a working solution you should be good with the newer version. – Cheslie

@Filburt: Maybe I want a quick and dirty way to deploy my web app? There doesn't seem to be a trivial way to deploy to the web server from the solution. Want to try trotting out an answer? You don't have to present a Robocopy-oriented answer. – Footlight

So, what's with the downvote? – Footlight

I didn't downvote but I admit I was tempted to point out that in the same time it took you to post your question you could have tried out yourself. – Cheslie

Oh, @Filburt, I know. I presently have six different things to do, all of which have to be finished before I can go home tonight. I was just trying to use StackOverflow as my gofer, in an attempt to save me some time. I was ALSO looking to see if there were a way to do this that I was not yet aware of. As I indicated, is there a better tool? I was perfectly willing to use whatever would be quick. Perhaps I am misusing my time. Thanks for your answer. – Footlight

But the next time someone has a question about Robocopy, well, guess what, @Filburt, the answer will be here. – Footlight

Answer 1

To answer all your questions:

Can I use ROBOCOPY for this?

Yes, RC should fit your requirements (simplicity, only copy what needed)

---

What exactly does it mean to exclude?

It will exclude copying - RC calls it skipping

---

Would the /XO option copy only newer files, not files of the same age?

Yes, RC will only copy newer files. Files of the same age will be skipped.

(the correct command would be robocopy C:\SourceFolder D:\DestinationFolder ABC.dll /XO)

---

Maybe in your case using the /MIR option could be useful. In general RC is rather targeted at directories and directory trees than single files.

Cheslie

Answer 2

You can use robocopy to copy files with an archive flag and reset the attribute. Use /M command line, this is my backup script with few extra tricks.

This script needs NirCmd tool to keep mouse moving so that my machine won't fall into sleep. Script is using a lockfile to tell when backup script is completed and mousemove.bat script is closed. You may leave this part out.

Another is 7-Zip tool for splitting virtualbox files smaller than 4GB files, my destination folder is still FAT32 so this is mandatory. I should use NTFS disk but haven't converted backup disks yet.

backup-robocopy.bat

@REM https://technet.microsoft.com/en-us/library/cc733145.aspx
@REM http://www.skonet.com/articles_archive/robocopy_job_template.aspx

set basedir=%~dp0
del /Q %basedir%backup-robocopy-log.txt

set dt=%date%_%time:~0,8%
echo "%dt% robocopy started" > %basedir%backup-robocopy-lock.txt
start "Keep system awake" /MIN /LOW  cmd.exe /C %basedir%backup-robocopy-movemouse.bat

set dest=E:\backup

call :BACKUP "Program Files\MariaDB 5.5\data"
call :BACKUP "projects"
call :BACKUP "Users\Myname"

:SPLIT
@REM Split +4GB file to multiple files to support FAT32 destination disk,
@REM splitted files must be stored outside of the robocopy destination folder.
set srcfile=C:\Users\Myname\VirtualBox VMs\Ubuntu\Ubuntu.vdi
set dstfile=%dest%\Users\Myname\VirtualBox VMs\Ubuntu\Ubuntu.vdi
set dstfile2=%dest%\non-robocopy\Users\Myname\VirtualBox VMs\Ubuntu\Ubuntu.vdi
IF NOT EXIST "%dstfile%" (
  IF NOT EXIST "%dstfile2%.7z.001" attrib +A "%srcfile%"
  dir /b /aa "%srcfile%" && (
    del /Q "%dstfile2%.7z.*"
    c:\apps\commands\7za.exe -mx0 -v4000m u "%dstfile2%.7z"  "%srcfile%"
    attrib -A "%srcfile%"
    @set dt=%date%_%time:~0,8%
    @echo %dt% Splitted %srcfile% >> %basedir%backup-robocopy-log.txt
  )
)

del /Q %basedir%backup-robocopy-lock.txt
GOTO :END


:BACKUP
TITLE Backup %~1
robocopy.exe "c:\%~1" "%dest%\%~1" /JOB:%basedir%backup-robocopy-job.rcj
GOTO :EOF


:END
@set dt=%date%_%time:~0,8%
@echo %dt% robocopy completed >> %basedir%backup-robocopy-log.txt
@echo %dt% robocopy completed
@pause

backup-robocopy-job.rcj

:: Robocopy Job Parameters
:: robocopy.exe "c:\projects" "E:\backup\projects" /JOB:backup-robocopy-job.rcj


:: Source Directory (this is given in command line)
::/SD:c:\examplefolder

:: Destination Directory (this is given in command line)
::/DD:E:\backup\examplefolder

:: Include files matching these names
/IF
    *.*

/M      :: copy only files with the Archive attribute and reset it.
/XJD    :: eXclude Junction points for Directories.

:: Exclude Directories
/XD
    C:\projects\bak
    C:\projects\old
    C:\project\tomcat\logs
    C:\project\tomcat\work
    C:\Users\Myname\.eclipse
    C:\Users\Myname\.m2
    C:\Users\Myname\.thumbnails
    C:\Users\Myname\AppData
    C:\Users\Myname\Favorites
    C:\Users\Myname\Links
    C:\Users\Myname\Saved Games
    C:\Users\Myname\Searches

:: Exclude files matching these names
/XF 
    C:\Users\Myname\ntuser.dat  
    *.~bpl

:: Exclude files with any of the given Attributes set
:: S=System, H=Hidden
/XA:SH      

:: Copy options
/S          :: copy Subdirectories, but not empty ones.
/E          :: copy subdirectories, including Empty ones.
/COPY:DAT   :: what to COPY for files (default is /COPY:DAT).
/DCOPY:T    :: COPY Directory Timestamps.
/PURGE      :: delete dest files/dirs that no longer exist in source.

:: Retry Options
/R:0        :: number of Retries on failed copies: default 1 million.
/W:1        :: Wait time between retries: default is 30 seconds.

:: Logging Options (LOG+ append)
/NDL        :: No Directory List - don't log directory names.
/NP         :: No Progress - don't display percentage copied.
/TEE        :: output to console window, as well as the log file.
/LOG+:c:\apps\commands\backup-robocopy-log.txt :: append to logfile

backup-robocopy-movemouse.bat

@echo off
@REM Move mouse to prevent maching from sleeping 
@rem while running a backup script

echo Keep system awake while robocopy is running,
echo this script moves a mouse once in a while.

set basedir=%~dp0
set IDX=0

:LOOP
IF NOT EXIST "%basedir%backup-robocopy-lock.txt" GOTO :EOF
SET /A IDX=%IDX% + 1
IF "%IDX%"=="240" (
  SET IDX=0
  echo Move mouse to keep system awake
  c:\apps\commands\nircmdc.exe sendmouse move 5 5
  c:\apps\commands\nircmdc.exe sendmouse move -5 -5
)
c:\apps\commands\nircmdc.exe wait 1000
GOTO :LOOP

Standee

9 year later, im sure you have a less-narcoleptic PC now... but in-case anyone else wonders - Microsoft "PowerToys" now includes what is commonly known as "caffeine" - a 1-click taskbar icon that prevents your computer sleeping. – Thomasthomasa

Answer 3

Looks like /e option is what you need, it'll skip same files/directories.

robocopy c:\data c:\backup /e

If you run the command twice, you'll see the second round is much faster since it skips a lot of things.

Chantay

According to: < computerhope.com/robocopy.htm > the /e is only responsible to "Copy subdirectories, including Empty ones". There is nothing about skipping files. – Dumpcart