PowerShell Script to Find and Replace for all Files with a Specific Extension

Question

I have several configuration files nested like such:

C:\Projects\Project_1\project1.config
  
C:\Projects\Project_2\project2.config

In my configuration I need to do a string replace like such:

<add key="Environment" value="Dev"/>

will become:

<add key="Environment" value="Demo"/>

I thought about using batch scripting, but there was no good way to do this, and I heard that with PowerShell scripting you can easily perform this. I have found examples of find/replace, but I was hoping for a way that would traverse all folders within my C:\Projects directory and find any files that end with the '.config' extension. When it finds one, I want it to replace my string values.

Any good resources to find out how to do this or any PowerShell gurus that can offer some insight?

Stacee

Let us know how you got on or if there were some odd formatting issues with the files that needed to be addressed. One good thing about the problem is that it's test without affecting production code – Wreath

Okay, I came here looking for PowerShell too, but I almost got suckered into taking up the "there was no good way to do this [in batch]" challenge. Luckily there are a few impressive cmd.exe/DOS style answers if you look far enough down on this question. – Ephrem

Answer 1

Here a first attempt at the top of my head.

$configFiles = Get-ChildItem . *.config -rec
foreach ($file in $configFiles)
{
    (Get-Content $file.PSPath) |
    Foreach-Object { $_ -replace "Dev", "Demo" } |
    Set-Content $file.PSPath
}

Wreath

Great stuff Brandon. I havn't fully embraced Powershell but when you consider how long this would take in VBScript!!!! – Wreath

For this to work in files in subdirectories, you need ".PSPath". Interestingly, when I tried to make this work without a () around get-content it failed at write-content because the file was locked. – Appulse

Short version (common aliases used): ls *.config -rec | %{ $f=$_; (gc $f.PSPath) | %{ $_ -replace "Dev", "Demo" } | sc $f.PSPath } – Structural

Using the get-content and set-content inside the same pipeline is giving me an error because the file locked (this seems like the obvious expected behaviour here). I've avoided the issue by storing the get-content result in a variable. Is there something I'm missing that would allow this to work otherwise? – Zenobiazeolite

You have to user bracket in (Get-Content $file.PSPath).. otherwise it remains opened. Same happened to me. – Stairwell

The search and replace pattern should be longer to prevent false positive matches. The string "Dev" could occur in other words or locations in the file. – Spiritualism

You might also get an UnauthorizedAccessException, in which case you might need to clear the ReadOnly flag like this: $configFiles | Set-ItemProperty -Name IsReadOnly -Value $false – Kunming

If you want to filter the files before the replace you can add the filter after the gci: | where-object {$_ | select-string "dev" -quiet} | – Madwort

@Structural don't forget the . after the ls. Got stung by that myself. – Aflame

There is no need for an extra foreach here. (Get-Content myFile ) -replace "cat", "dog" will replace all occurences of "cat" by "dog" – Hemelytron

UnauthorizedAccessException may also cause due to folders if you will remove the *.config to run on all files. You can add -File filter to the Get-ChildItem... Took a while to figure it out – Rattler

Careful!! The -replace operator uses regex, so you'll get some unexpected results if you're trying to replace any symbols (like "*"). This burned me just now. – Kinnikinnick

This is a good, practical answer, but, unfortunately, it adds a newline at the end of the file if there wasn't one already (possible solution here) and removes the BOM if the file uses one. Looks like adding -Encoding utf8BOM to Set-Content may fix that (if you're using UTF-8 with a BOM, natch). YMMV if these are things to worry about or not. – Ephrem

This can unintentionally change line endings to Windows' style. For UNIX style line endings (LF), with the common alias form: ls *.config -rec | %{ $f=$_; ((gc $f.PSPath) -join "`n") + "`n" | %{ $_ -replace "Dev", "Demo" } | sc $f.PSPath -NoNewLine } . Credit to mklement0 from SO answer. WARNING: There's a file size limit. Read the caveats from mklement0's answer. – Changeup

Answer 2

PowerShell is a good choice ;) It is very easy to enumerate files in given directory, read them and process.

The script could look like this:

Get-ChildItem C:\Projects *.config -recurse |
    Foreach-Object {
        $c = ($_ | Get-Content) 
        $c = $c -replace '<add key="Environment" value="Dev"/>','<add key="Environment" value="Demo"/>'
        [IO.File]::WriteAllText($_.FullName, ($c -join "`r`n"))
    }

I split the code to more lines to be readable for you. Note that you could use Set-Content instead of [IO.File]::WriteAllText, but it adds new line at the end. With WriteAllText you can avoid it.

Otherwise the code could look like this: $c | Set-Content $_.FullName.

Isabellisabella

With the most voted answer, I was having issues with the files being used by another process (even with the parenthesis around Get-Content) and this solution fixed it. I believe it happened because the way I wrote the script I didn't have access to PSPath (but I am not entirely sure). These files are blocked by p4 and I was calling 'p4 edit' on them before editing. I used the output of Select-String to get the files list first, so I would only call 'p4 edit' on those specific files. – Arpeggio

Answer 3

This approach works well:

gci C:\Projects *.config -recurse | ForEach {
  (Get-Content $_ | ForEach {$_ -replace "old", "new"}) | Set-Content $_ 
}

• Change "old" and "new" to their corresponding values (or use variables).
• Don't forget the parenthesis -- without which you will receive an access error.

Martijn

So I went with this one for succinct expression - but I had to replace Get-Content $_ with Get-Content $_.FullName, and the equivalent for Set-Content for it to handle files that weren't at the root. – Odetteodeum

Answer 4

This powershell example looks for all instances of the string "\foo\" in a folder and its subfolders, replaces "\foo\" with "\bar\" AND DOES NOT REWRITE files that don't contain the string "\foo\" This way you don't destroy the file last update datetime stamps where the string was not found:

Get-ChildItem  -Path C:\YOUR_ROOT_PATH\*.* -recurse 
 | ForEach {If (Get-Content $_.FullName | Select-String -Pattern '\\foo\\') 
           {(Get-Content $_ | ForEach {$_ -replace '\\foo\\', '\bar\'}) | Set-Content $_ }
           }

Hunger

Answer 5

I found @Artyom's comment useful but unfortunately they have not posted it as an answer.

This is the short version, in my opinion a better one, of the accepted answer;

ls *.config -rec | %{$f=$_; (gc $f.PSPath) | %{$_ -replace "Dev", "Demo"} | sc $f.PSPath}

Quinze

In case anyone else runs across this, as I did -- looking to execute this directly from a batch file -- It may help to use foreach-object instead of the % alias when executing a command like this. Otherwise, it may result in the error: Expressions are only allowed as the first element of a pipeline – Kannry

Answer 6

I would go with xml and xpath:

dir C:\Projects\project_*\project*.config -recurse | foreach-object{  
   $wc = [xml](Get-Content $_.fullname)
   $wc.SelectNodes("//add[@key='Environment'][@value='Dev']") | Foreach-Object {$_.value = 'Demo'}  
   $wc.Save($_.fullname)  
}

Rida

Answer 7

I have written a little helper function to replace text in a file:

function Replace-TextInFile
{
    Param(
        [string]$FilePath,
        [string]$Pattern,
        [string]$Replacement
    )

    [System.IO.File]::WriteAllText(
        $FilePath,
        ([System.IO.File]::ReadAllText($FilePath) -replace $Pattern, $Replacement)
    )
}

Example:

Get-ChildItem . *.config -rec | ForEach-Object { 
    Replace-TextInFile -FilePath $_ -Pattern 'old' -Replacement 'new' 
}

Brocade

Answer 8

When doing recursive replacement, the path and filename need to be included:

Get-ChildItem -Recurse | ForEach {  (Get-Content $_.PSPath | 
ForEach {$ -creplace "old", "new"}) | Set-Content $_.PSPath }

This wil replace all "old" with "new" case-sensitive in all the files of your folders of your current directory.

Celadon

The ".PSPath" part of your answer really helped me. But I had to change the inner "{$" to "$_". I'd edit your answer, but I'm not using your -creplace part--I'm using the accepted answer with .PSPath – Character

Answer 9

Some of the previous answers will convert line endings to Windows-style CRLF regardless whether they were originally UNIX-style LF and may also corrupt files with an unexpected encoding.

The -Replace operator users regular expressions. The .Replace method uses literal text.

(I work with both Windows and UNIX line endings but have standardized to UTF8 without a BOM.)

Specifying the encoding, trapping read errors, and using the -Raw flag to grab all the lines at once (with their delimiters) will mitigate these issues.

$utf8 = New-Object Text.UTF8Encoding
$files = Get-ChildItem -Path 'test1.sql', 'test2.sql'
$find = 'an old needle from our haystack'
$replace = 'a new needle for our haystack'

ForEach ($f In $files) {
   $text_content = $Null
   $text_content = Get-Content -PSPath $f.PSPath -Raw -Encoding utf8 -ErrorAction SilentlyContinue
   If ($text_content) {
      $text_content = $text_content.Replace($find, $replace)
      $byte_content = $utf8.GetBytes($text_content)
      Set-Content -Value $byte_content -Encoding Byte -PSPath $f.PSPath
   }
}

Triglyph