PowerShell Remove-Item not waiting

Question

If have this piece of code

if(Test-Path -Path $OUT) 
{ 
    Remove-Item $OUT -Recurse 
}
New-Item -ItemType directory -Path $OUT

Sometimes it works, but sometimes the New-Item line produces a PermissionDenied ItemExistsUnauthorizedAccessError error. Which, I assume, means that the previous Remove-Item was not completely performed yet and the folder cannot be created because it still exists.

If I insert a sleep there

if(Test-Path -Path $OUT) 
{ 
    Remove-Item $OUT -Recurse 
    Start-Sleep -s 1
}
New-Item -ItemType directory -Path $OUT

then it works always. How can I force Remove-Item to ensure that the folder is really removed? Or maybe I miss something else?

Paschasia

Answer 1

The Remove-Item command has a known issue.

Try this instead:

if (Test-Path $OUT) 
{ 
    # if exists: empty contents and reuse the directory itself
    Get-ChildItem $OUT -Recurse | Remove-Item -Recurse
}
else
{
    # else: create
    New-Item -ItemType Directory -Path $OUT
}

Note:

• The Get-ChildItem command only finds non-hidden files and subdirectories, so emptying out the target directory may not be complete; to include hidden items too, add -Force.

• Similarly, add -Force to -RemoveItem to force removal of files that have the read-only attribute set.

  • Without -Force, emptying may again be incomplete, but you'll get non-terminating errors in this case; if you want to treat them as terminating errors, add -ErrorAction Stop too.

Maryettamaryjane

Answer 2

Update: Starting with (at least ^[1]) Windows 10 version 20H2 (I don't know that Windows Server version and build that corresponds to; run winver.exe to check your version and build), the DeleteFile Windows API function now exhibits synchronous behavior, which implicitly solves the problems with PowerShell's Remove-Item and .NET's System.IO.File.Delete / System.IO.Directory.Delete (but, curiously, not with cmd.exe's rd /s).

---

Remove-Item -Recurse is unexpectedly asynchronous, ultimately because the Windows API methods for file and directory removal are inherently asynchronous and Remove-Item doesn't account for that.

This intermittently, unpredictably manifests in one of two ways:

• Your case: recreating a removed directory immediately after removal can fail, because the removal may not have completed yet by the time re-creation is attempted.

• More typically: Removing a nonempty directory itself can fail, if removal of a subdirectory or file hasn't completed yet by the time an attempt is made to remove the parent directory - this is demonstrated in the ServerFault answer marsze links to.

A potential workaround is to reuse an existing directory by emptying it - instead of deleting and recreating it.

However, emptying the directory with Get-ChildItem $OUT -Recurse | Remove-Item -Recurse is also susceptible to intermittent failures, though likely less often.

The problem not only affects PowerShell's Remove-Item, but also cmd.exe's rd /s as well as .NET's [System.IO.Directory]::Delete():

As of Windows PowerShell v5.1 / PowerShell Core 6.2.0-preview.1 / cmd.exe 10.0.17134.407 / .NET Framework 4.7.03056, .NET Core 2.1, neither Remove-Item, nor rd /s, nor [System.IO.Directory]::Delete() work reliably, because they fail to account for the asynchronous behavior of the Windows API file/directory-removal functions:

• PowerShell Core bug report
• cmd.exe bug report
• .NET Core bug report

For a custom PowerShell function that provides a reliably synchronous workaround, see this answer.

---

^{[1] I've personally verified that the issue is resolved in version 20H2, by running the tests in GitHub issue #27958 for hours without failure; this answer suggests that the problem was resolved as early as version 1909, starting with build 18363.657, but Dinh Tran finds that the issue is not resolved as of build 18363.1316 when removing large directory trees such as node_modules. I couldn't find any official information on the subject.}

Comprehensible

Answer 3

For completeness' sake: You can also use the safe and fast .NET methods:

if ([System.IO.Directory]::Exists($OUT)) {
    [System.IO.Directory]::Delete($OUT, $true)
}
[System.IO.Directory]::CreateDirectory($OUT)

Note:

Depending on where you get the value of $OUT you might want to convert it to a full path first to make sure the .NET methods remove the correct directory (see @mklement0`s comment):

$fullPath = Convert-Path $OUT

Maryettamaryjane

Nice alternative, but I strongly suggest passing (Convert-Path $OUT) rather than just $OUT, because the .NET framework's notion of the working directory typically differs from PowerShell's, so with a relative path in $OUT the target directory may not be found or, worse, a different directory may get deleted. – Comprehensible

I use [System.IO.Directory]::SetCurrentDirectory((Get-Location).Path) in many of my custom cmdlets which is extremely useful when interacting with any dlls. – Maryettamaryjane

That is fine if you know you're the only runspace in the process (which is typically true in an interactive shell); the possibility of multiple runspaces is the reason why the directories aren't synced to begin with - see github.com/PowerShell/PowerShell/issues/3428. Note that it's better to use .ProviderPath rather than .Path, because .Path can be based on PS drives, which .NET is unaware of. – Comprehensible

@Comprehensible All true. Nevertheless, I typically choose to ignore such advanced considerations for edge cases, for simplicity, just as I don't bother to make all of my classes perfectly threadsafe. If OP struggles with removing a directory he'll surely not be doing very advanced stuff with his scripts. – Maryettamaryjane

As it turns out, [System.IO.Directory]::Delete() is buggy too: github.com/dotnet/corefx/issues/33603 – Comprehensible

Answer 4

If you type Get-Help Remove-Item -Detailed you'll see:

Example 4: Delete files in subfolders recursively
PS C:\>Get-ChildItem * -Include *.csv -Recurse | Remove-Item

This command deletes all of the CSV files in the current folder and all subfolder recursively.

Because the Recurse parameter in Remove-Item has a known issue, the command in this example uses Get-ChildItem to get the desired files, and then uses the pipeline operator to pass them to Remove-Item .

Do what specification recommends:

if(Test-Path -Path $OUT) 
{ 
    Get-ChildItem $OUT -Recurse | Remove-Item
}
New-Item -ItemType directory -Path $OUT

Leuctra

The bug you're referencing is a different one; it relates to the -Include parameter, which does not come into play here. Your workaround doesn't delete the target directory itself, so if you want to call New-Item unconditionally without causing an error, you must add -Force to it. -Recurse should be added to Remove-Item, because you'll otherwise get a confirmation prompt for every subdir in the target directory's subtree. – Comprehensible

To make it clearer that the referenced bug is unrelated: the-Recurse parameter's description in the same help topic states more explicitly (emphasis added), "When it is used with the Include parameter, the Recurse parameter might not delete all subfolders or all child items. This is a known issue. " – Comprehensible