Invoke-Command faster than the command itself?

Question

I was trying to measure some ways to write to files in PowerShell. No question about that but I don't understand why the first Measure-Command statement below takes longer to be executed than the 2nd statement.

They are the same but in the second one I write a scriptblock to send to Invoke-Command and in the 1st one I only run the command.

All informations about Invoke-Command speed I can find are about remoting.

This block takes about 4 seconds:

Measure-Command {
    $stream = [System.IO.StreamWriter] "$PSScriptRoot\t.txt"
    $i = 0
    while ($i -le 1000000) {
        $stream.WriteLine("This is the line number: $i")
        $i++
    }
    $stream.Close() 
} # takes 4 sec

And this code below which is exactly the same but written in a scriptblock passed to Invoke-Command takes about 1 second:

Measure-Command {
    $cmdtest = {
        $stream = [System.IO.StreamWriter] "$PSScriptRoot\t2.txt"
        $i = 0
        while ($i -le 1000000) {
            $stream.WriteLine("This is the line number: $i")
            $i++
        }
        $stream.Close()
     }
     Invoke-Command -ScriptBlock $cmdtest
} # Takes 1 second

How is that possible?

Floweret

Might be because icm doesn't create a steppable pipeline in this particular case. Anyway, this is an awesome find! – Scarberry

Ben, @wOxxOm: Oops! My answer was originally not quite framed correctly - the issue is unrelated to steppable pipelines. It's only about dot-sourcing vs. running in a child scope - please see my update. – Ezzell

Indeed, I even occasionally used re-assigning to variables with the same name in a heavy function to make them local. – Scarberry

Answer 1

As it turns out, based on feedback from a PowerShell team member on GitHub issue #8911, the issue is more generally about (implicit) dot-sourcing (such as direct invocation of an expression) vs. running in a child scope, such as with &, the call operator, or, in the case at hand, with Invoke-Command -ScriptBlock.

Running in a child scope avoids variable lookups that are performed when (implicitly) dot-sourcing.

Therefore, as of Windows PowerShell v5.1 / PowerShell (Core) 7.2.x, you can speed up statements involving script blocks by invoking them via & { ... }, in a child scope (somewhat counter-intuitively, given that creating a new scope involves extra work).

Note that using & means that such blocks then cannot modify the caller's variables directly, but there are workarounds.

The following simplified code, which uses a foreach expression to loop 1 million times (1e6) demonstrates the performance advantage of running via & { ... }:

# REGULAR, direct invocation of an expression (a `foreach` statement in this case), 
# which is implicitly DOT-SOURCED
(Measure-Command { $result = foreach ($n in 1..1e6) { $n } }).TotalSeconds

# OPTIMIZED invocation in CHILD SCOPE, using & { ... }
# up to 10+ TIMES FASTER, depending on OS and PowerShell edition
(Measure-Command { $result = & { foreach ($n in 1..1e6) { $n } } }).TotalSeconds

However, note that the performance advantage diminishes and can even go away the more preexisting variables are being referenced in the script block:

# Define a few sample variables to reference in the script blocks.
# Note that, due to PowerShell's dynamic scoping, even the child
# scope created by & { ... } sees these variables.
$i1=1; $i2=2; $i3=3; $i4=4; $i5=5

(Measure-Command { $result = foreach ($n in 1..1e6) { $n, $i1, $i2, $i3, $i4, $i5 } }).TotalSeconds

# MAY OR MAY NOT BE FASTER, depending on the OS and PowerShell edition.
(Measure-Command { $result = & { foreach ($n in 1..1e6) { $n, $i1, $i2, $i3, $i4, $i5 } } }).TotalSeconds

The reason is that variables that aren't created in the script block (by assigning to them inside it) require a variable lookup with & { ... } too, due to PowerShell's dynamic scoping (see this answer).

Ezzell