ROBOCOPY - Copy folders content to a single folder

Question

Here is some code I made :)

@echo off
set source="R:\Contracts\"
set destination="R:\Contracts\Sites\"
ROBOCOPY %source% %destination% *.srt *.pdf *.mp4 *.jpg /COPYALL /R:0 /S
for /r %source in (*) do @copy "%destination" .

R:\Contracts\ is full of folders which have files in them.

I want to copy all to R:\Contracts\Sites\ and flatten the folder structure.

Everything copies well but also the folder structure.

Thank you

Salangi

possible duplicate of How to copy files from folder tree dropping all the folders with Robocopy? – Biblicist

Answer 1

No single command will flatten the hierarchy for you; you will have to use multiple commands. It can be done simply by using FOR /R to walk the hierarchy, coupled with your copy/move command of choice (move, copy, xcopy, robocopy). Because your destination is within the source hierarchy, you need an IF to prevent the destination from being a source.

Before proceeding you should stop and think about what happens if the same file name appears in multiple source folders. You can only have one version in your destination folder. Can you guarantee no duplicate names exist? If not, which file should be kept? How can you structure the command to keep the file you want? This complication is probably why no command was ever written to simply flatten a hierarchy.

Here is your ROBOCOPY command integrated with the FOR /R solution.

@echo off
set source="R:\Contracts"
set destination="R:\Contracts\Sites"

::Not sure if this is needed
::It guarantees you have a canonical path (standard form)
for %%F in (%destination%) do set destination="%%~fF"

for /r %source% %%F in (.) do if "%%~fF" neq %destination% ROBOCOPY "%%F" %destination% *.srt *.pdf *.mp4 *.jpg /COPYALL /R:0

Foxy

Hello dbenham, first of all, Thank You VERY MUCH. I have tried the above code however it does not seem to work. I have made a batch file and tested it. I have been through the code and cannot see any direct mistake. Please let me know. Thank you – Salangi

Hello, I removed this line "for %%F in (%destination%) do set destination="%~fF"" and it seems to work. However I am not sure if it is needed. If thank Thank you DBENHAM – Salangi

Oops - I was missing one % near the end of that line. It's fixed now. It's purpose was to make sure the path was represented in a fully qualified, standard form so that the subsequent IF statement works properly. – Foxy

Hello, I have a question please. When I set the source and destination with a short file name it all works however if I user a long files name e.g. "c:\files\contract open\site\usa\ content clean\over load\psjj200" it does not seem to work. Any help please – Salangi

@Salangi - I forgot to put quotes around the ROBOCOPY source ("%%F"). Edit completed. – Foxy

So, how does this handle repeat file names? Which file is kept? – Figurehead

@ColeTrumbo - Duplicates are not handled well. This solution should only be used if you are confident there are no duplicate file names. If you run the above with duplicates, then the last one copied "wins". – Foxy

Good solution, however for the large structures (many subfolders) it is inefficient. ROBOCOPY reviews destination folder every time, so if you have, say 5000 subfolders and 10000 files you will have to wait 5000 times for review of 5000 files on average. In my case it lasts days... – Length

Answer 2

You could do that with a PowerShell one liner. In this example I filter out all the files with the .txt extension from all the subfolders. And then send them to the Copy-Item Cmdlet.

Combine the Cmdlets Get-Childitem (GCI for short), -recurse, and -filter and then pipe the result to the Copy-Item Cmdlet. Use -WhatIf first to check that the output is what you expected.

Copy to another folder (Use -WhatIf and verify the output to check your command before committing to copying the files):

Get-Childitem -recurse R:\Contracts -filter *.txt | Copy-Item -Destination R:\Contracts\Sites -WhatIf

To do multiple filetypes as you've asked, you can just run multiple commands, one for each filetype.

Exophthalmos

brilliant. simple, easy to understand. – Health

downvoted because Move-Item should be changed to Copy-Item. Change and I will upvote – Tar

Edited to use Copy-Item and the source/destination from the original post. – Arrear

Answer 3

Similar to the previous Powershell option, I did the following to flatten a multi-subdirectory music folder:

#Get all files and not the directories
$files = Get-ChildItem -Path R:\Contracts -Recurse | Where {$_.PSIsContainer -eq $false}

#Copy items from sources to new destination
foreach ($file in $files){
    Copy-Item -Path $file.FullName -Destination R:\Contracts\Sites\$($file.Name)
}

The Get-ChildItem with the -Recurse switch will get a listing of all sub-folders and files. The Where function is stripping out any directories by checking the boolean PSIsContainer property. Without stripping the directories the sub-folder structure would be created without files in them. This listing is stored in the $files variable.

The foreach function runs through the list of files in the $files variable and stores one item at a time in the $file variable. The Copy-Item function then uses the full path from $file.FullName and then copies the file to the destination with the same name from $file.Name.

Jape

Answer 4

I recently had to tackle this problem, and many files that I wanted to move to from the hierarchy to a single folder had the same name as each other, and I wanted to still flatten the hierarchy without them to being over-written. What I did was write a script that moves the file, but renames it with the old hierarchy path in the name for example: source files:

C:\files\somefiles\file.txt

C:\files\otherfiles\file.txt

destination is C:\newdir\ files are created as

C:\newdir\somefiles-file.txt

C:\newdir\otherfiles-file.txt

here is the code, batch file 1 goes thru the files, batch file 2 renames and moves them (could also copy instead, if you want to preserve the source:

@echo off
for /r %%f in (*.*pr) do @renameandmovefilespart2.bat "%%f" "%%~ff" "%%~xf"

renameandmovefilespart2.bat

@echo off
Setlocal EnableDelayedExpansion
rem set the whole file path
set origWhole=%1
set origPathOnly=%2
set extension=%3
rem here you can set where the directory to hold the flattened hierarchy is
set destDir=c:\destinationDir\
rem  set the directory to do a string replace
rem make this the starting directory, that you dont want in the newly renamed files
set startingDir=C:\starting\directory\
set nothing=
set slash=\
rem here you can set what the character to represent the directory indicator \ in the new files 
set reaplcementDirectoryCharacter=--
set quote="
rem cut out the starting part of the directory
call set newname=%%origWhole:!startingDir!=!nothing!%%
rem replace slashes with new character
call set newname=%%newname:!slash!=!reaplcementDirectoryCharacter!%%
rem remove quotes
call set newname=%%newname:!quote!=!nothing!%%
rem @echo shortened: %newname%
rem @echo source path: %origPathOnly% newPath: %startingDir%
rem @echo extension: %extension%
rem rename the files
ren %origWhole% %newname%
rem prepare to move the file, clean up the source path
call set origPathOnly=%%origPathOnly:!quote!=!nothing!%%
move "%origPathOnly%%newname%" "%destDir%"

Paronomasia

Answer 5

There's an old PCMag utility called Sweep.exe that can operate the same command in the current and subdirectory. I wouldn't put the destination as a subdirectory of the source directory. Put the destination elsewhere.

http://www.rarewares.org/files/case/Sweep.zip

cd c:\contracts sweep copy *.* c:\sites

This will copy everything from c:\contracts and underneath to c:\sites

I use a similar command to flatten the hierarchy of a directory.

Take care with duplicates and how you want to handle them. Do you want to overwrite or handle a different way.

Hindsight

Answer 6

For those familiar with R copying files to a single folder could be easily scripted as follow. It is more efficient than ROBOCOPY for large sets and directory structures.

## R script - Flatten files in folder structure
# declare your paths
sourcepath<-C:\\Myfolder\\PicturesInStructure"
destpath<-"C:\\Myfolder\\PicturesTogether\\"
# read source filenames with path to vector variable
fileslist<-list.files(sourcepath,
                      pattern="*.jpg",    # in this example copies JPG pictures
                      full.names=TRUE,    # retrieves full path
                      recursive=TRUE,     # runs through all subdirs
                      include.dirs=FALSE) # skips dir names (no to be copied)
# here you will get a head of your list
head(fileslist)
# ordinary loop by source filenames
for (i in 1:length(fileslist)) {
  # Number of Slashes required to extract filename from the source path
  NoS<-lengths(regmatches(fileslist[i], gregexpr("/", fileslist[i])))
  # for lengthy processes it is good to know where you are with this 
  cat(i,fileslist[i],"                       \r")
  # copy one file to a destination folder (full path composed with paste0())
  file.copy(fileslist[i],paste0(destpath,unlist(strsplit(fileslist[i],"/"))[NoS+1]))
}

Length