Concatenate files using PowerShell

Question

I am using PowerShell 3.

What is best practice for concatenating files?

file1.txt + file2.txt = file3.txt

Does PowerShell provide a facility for performing this operation directly? Or do I need each file's contents be loaded into local variables?

Abstemious

Possible duplicate of How do I concatenate two text files in PowerShell? – Menchaca

Answer 1

If all the files exist in the same directory and can be matched by a simple pattern, the following code will combine all files into one.

Get-Content .\File?.txt | Out-File .\Combined.txt

Sewellel

Get-Content ... | Out-File ... is arguably the clearest expression of my intent when I'm doing something like this. It doesn't matter to me whether I can match a simple pattern; there are a lot of ways to select files in PowerShell. The default encoding for Out-File is more often correct for me than the default encoding for Set-Content; Out-File defaults to Unicode, and Set-Content defaults to ASCII. – Parclose

For everyone wondering why this fails on Windows 8+, use this instead: Get-Content .\*.txt | Out-File .\Combined.txt – Dodecasyllable

Like @masi says, I was confused by the pattern expression. For everyone as dense as me, the question mark (?) only matches one character. Use an asterisk (*) if you need to match multiple. – Hospitable

That produced a recursive file bomb for me, don't put the output file in the same directory as the input files. – Sidran

Not very useful because it doesn't show that input files must be separated by commas. – Rosalindarosalinde

The outfile will be UTF-16, which doubles the file size – Okajima

Answer 2

I would go this route:

Get-Content file1.txt, file2.txt | Set-Content file3.txt

Use the -Encoding parameter on Set-Content if you need something other than ASCII which is the default for Set-Content.

Mcneese

+1, I always love these simple one's :) Of course, Out-File would be just as good. – Bini

@AthomSfere Yes and if you want Unicode encoding Out-File defaults to that. – Mcneese

This answer avoids the problem of recursion when *.txt is used as input (because set-content locks the file) – Renaud

Answer 3

If you need more flexibility, you could use something like

Get-ChildItem -Recurse *.cs | ForEach-Object { Get-Content $_ } | Out-File -Path .\all.txt

Andras

Answer 4

Warning: Concatenation using a simple Get-Content (whether or not using -Raw flag) works for text files; Powershell is too helpful for that:

• Without -Raw, it "fixes" (i.e. breaks, pun intended) line breaks, or what Powershell thinks is a line break.
• With -Raw, you get a terminating line end (normally CR+LF) at the end of each file part, which is added at the end of the pipeline. There's an option for that in newer Powershells' Set-Content.

To concatenate a binary file (that is, an arbitrary file that was split for some reason and needs to be put together again), use either this:

Get-Content -Raw file1, file2 | Set-Content -NoNewline destination

or something like this:

Get-Content file1 -Encoding Byte -Raw | Set-Content destination -Encoding Byte
Get-Content file2 -Encoding Byte -Raw | Add-Content destination -Encoding Byte

An alternative is to use the CMD shell and use

copy file1 /b + file2 /b + file3 /b + ... destinationfile

You must not overwrite any part, that is, use any of the parts as destination. The destination file must be different from any of the parts. Otherwise you're up for a surprise and must find a backup copy of the file part.

Mcintire

On PowerShell 5, it failed trying to concat together a few 2GB files. But CMD copy worked great! – Promote

Answer 5

gc file1.txt, file2.txt > output.txt

I think this is as short as it gets.

Diphtheria

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Effusion

Answer 6

In case you would like to ensure the concatenation is done in a specific order, use the Sort-Object -Property <Some Name> argument. For example, concatenate based on the name sorting in an ascending order:

Get-ChildItem -Path ./* -Include *.txt -Exclude output.txt | Sort-Object -Property Name | ForEach-Object { Get-Content $_ } | Out-File output.txt

IMPORTANT: -Exclude and Out-File MUST contain the same values, otherwise, it will recursively keep on adding to output.txt until your disk is full.

Note that you must append a * at the end of the -Path argument because you are using -Include, as mentioned in Get-ChildItem documentation.

Rotten

Answer 7

a generalization based on @Keith answer:

gc <some regex expression> | sc output

Nauplius

while this answer looks like it meant bash, the reason for short names are aliases in PowerShell - where gc is Get-Content and sc is Set-Content. – Purdah

Answer 8

Here is an interesting example of how to make a zip-in-image file based on Powershell 7

Get-Content -AsByteStream file1.png, file2.7z | Set-Content -AsByteStream file3.png
Get-Content -AsByteStream file1.png, file2.7z | Add-Content -AsByteStream file3.png

Benedikt

Answer 9

None of the examples above worked very well for me because I was dealing with large (5GB+) files. Set-Content makes it first be all loaded in memory (explodes RAM use) and then write it all. Instead, a text stream write is better. I put the below approach to combine a header file and a data file after looking at the answers to this other question!

$file = [system.io.file]::OpenWrite("$($pwd.Path)\out.txt")
$writer = New-Object System.IO.StreamWriter($file)

cat headers.txt,data.txt | ForEach-Object { $writer.WriteLine($_) }

$writer.Close()
$file.Close()

Expedite