How do I retain ScriptStackTrace in an exception thrown from within an Invoke-Command on a remote computer?

Question

I'm writing a Powershell script which executes one of the steps in my build/deploy process, and it needs to run some actions on a remote machine. The script is relatively complex, so if an error occurs during that remote activity I want a detailed stack trace of where in the script the error occurred (over and above the logging that is already produced).

The problem arises in that Invoke-Command loses stack trace information when relaying terminating exceptions from a remote machine. If a script block is invoked on the local machine:

Invoke-Command -ScriptBlock {
    throw "Test Error";
}

The required exception detail is returned:

Test Error
At C:\ScriptTest\Test2.ps1:4 char:2
+     throw "Test Error";
+     ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Test Error:String) [], RuntimeException
    + FullyQualifiedErrorId : Test Error

But if run remotely:

Invoke-Command -ComputerName $remoteComputerName -ScriptBlock {
    throw "Test Error";
}

The exception stack trace points to the whole Invoke-Command block:

Test Error
At C:\ScriptTest\Test2.ps1:3 char:1
+ Invoke-Command -ComputerName $remoteComputerName -ScriptBlock {
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Test Error:String) [], RuntimeException
    + FullyQualifiedErrorId : Test Error

I can transport the exception back to the local machine manually:

$exception = Invoke-Command -ComputerName $remoteComputerName -ScriptBlock {
    try
    {
        throw "Test Error";
    }
    catch
    {
        return $_;
    }
}

throw $exception;

But re-throwing it loses the stack trace:

Test Error
At C:\ScriptTest\Test2.ps1:14 char:1
+ throw $exception;
+ ~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Test Error:PSObject) [], RuntimeException
    + FullyQualifiedErrorId : Test Error

If I write the exception to Output:

$exception = Invoke-Command -ComputerName $remoteComputerName -ScriptBlock {
    try
    {
        throw "Test Error";
    }
    catch
    {
        return $_;
    }
}

Write-Output $exception;

I get the correct stack trace information:

Test Error
At line:4 char:3
+         throw "Test Error";
+         ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Test Error:String) [], RuntimeException
    + FullyQualifiedErrorId : Test Error

But as it's not on the Error stream it isn't picked up correctly by my build tools. If I try Write-Error, I have a similar problem to re-throwing the exception and the stack trace points to the wrong part of the script.

So my question is - how do I get Powershell to report the exception from a remote machine as if it had been raised locally, with the same stack trace information and on the Error stream?

Intermit

Use something like Invoke-Command -ErrorVariable remoteerror then the results of the error stream should be in $remoteerror? – Thistledown

I take it back -ErrorVariable should not contain the information you need. I think I have an idea thought using throw – Thistledown

Is anything on this article helpful? "For debugging, instead on Invoke-Command, it might be better to use Enter-PSSession, so that you can have a shell on the remote machine to try things out" – Nonary

You can't use Enter-PSSession in scripts – Lamonica

Answer 1

When you run some code and it fails, you receive an ErrorRecord that reflects the code you (local computer) executed. So when you use throw "error" you can access the invocationinfo and exception for that code.

When you use Invoke-Command, you are not executing throw "error" anymore, the remote computer is. You (local computer) are executing Invoke-Command ...., which is why the ErrorRecord you get reflects that (and not the real exception like you wanted). This is the way it has to be since an exception may be coming from the scriptblock the remote comptuer executed, but it could just as well be an exception from Invoke-Command itself because it couldn't connect to the remote computer or something similar.

When the exception is originally thrown on the remote computer, Invoke-Command/PowerShell throws a RemoteException on the local computer.

#Generate errors
try { Invoke-Command -ComputerName localhost -ScriptBlock { throw "error" } }
catch { $remoteexception = $_ }

try { throw "error" }
catch { $localexception = $_ }

#Get exeception-types
$localexception.Exception.GetType().Name
RuntimeException

$remoteexception.Exception.GetType().Name
RemoteException

This exception-type has a few extra properties, including SerializedRemoteException and SerializedRemoteInvocationInfo which contains the information from the exception that was thrown in the remote session. Using these, you can receive the "internal" exception.

• SerializedRemoteException: Gets the original exception that was thrown by the remote instance of Windows PowerShell.
• SerializedRemoteInvocationInfo: Gets the invocation information for the remote instance of Windows PowerShell.

Sample:

#Show command that threw error 
$localexception.InvocationInfo.PositionMessage

At line:4 char:7
+ try { throw "error" }
+       ~~~~~~~~~~~~~

$remoteexception.Exception.SerializedRemoteInvocationInfo.PositionMessage    

At line:1 char:2
+  throw "error"
+  ~~~~~~~~~~~~~

You can then write a simple function to extract the information dynamically, ex:

function getExceptionInvocationInfo ($ex) {
    if($ex.Exception -is [System.Management.Automation.RemoteException]) {
        $ex.Exception.SerializedRemoteInvocationInfo.PositionMessage
    } else {
        $ex.InvocationInfo.PositionMessage
    }
}

function getException ($ex) {
    if($ex.Exception -is [System.Management.Automation.RemoteException]) {
        $ex.Exception.SerializedRemoteException
    } else {
        $ex.Exception
    }
}

getExceptionInvocationInfo $localexception

At line:4 char:7
+ try { throw "error" }
+       ~~~~~~~~~~~~~

getExceptionInvocationInfo $remoteexception
At line:1 char:2
+  throw "error"
+  ~~~~~~~~~~~~~

Be aware that the SerializedRemoteExpcetion is shown as PSObject because of the serialization/deserialization during network transfer, so if you're going to check the exception-type you need to extract it from psobject.TypeNames.

$localexception.Exception.GetType().FullName
System.Management.Automation.ItemNotFoundException

$remoteexception.Exception.SerializedRemoteException.GetType().FullName
System.Management.Automation.PSObject

#Get TypeName from psobject
$remoteexception.Exception.SerializedRemoteException.psobject.TypeNames[0]
Deserialized.System.Management.Automation.ItemNotFoundException

Lamonica

Great - this gives me all the information from the exception that I need. Thanks! – Intermit

@FrodeF. Fantastic answer. I can get the PositionMessage, but I cannot seem to get a StackTrace or a ScriptStackTrace from the serialized exception. Any ideas? – Cary

The scriptstacktrace is part of the errorrecord, not the exception. $remoteexception.ScriptStackTrace works fine for me. As for the exception stacktrace, I get the same output in $remoteexception.Exception.SerializedRemoteException.InnerException.StackTrace as I do from $localexception.Exception.InnerException.StackTrace. The outer exception stacktrace doesn't match in my test, but that's just because Invoke-Command required -ErrorAction Stop to generate the exception as terminating. – Lamonica

Answer 2

I am sure someone with more experience can help but I would like to give you something to chew on in the mean time. Sounds like you want to be using throw since you are looking for terminating exceptions. Write-Error does write to the error stream but it is not terminating. Regardless of your choice there my suggestion is still the same.

Capturing the exception into a variable is a good start for this so I would recommend this block from your example:

$exception = Invoke-Command -ComputerName $remoteComputerName -ScriptBlock {
    try
    {
        throw "Test Error";
    }
    catch
    {
        return $_;
    }
}

$exception in this case should be a Deserialized.System.Management.Automation.ErrorRecord. You can send custom objects to throw...

You can also throw an ErrorRecord object or a Microsoft .NET Framework exception.

but in this case it does not work which is likely due to the deserialization. At one point I tried to create my own error object but some of the needed properties were read only so I skipped that.

PS M:\Scripts> throw $exception
Test Error
At line:1 char:1
+ throw $return
+ ~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Test Error:PSObject) [], RuntimeException
    + FullyQualifiedErrorId : Test Error

However just writing to the output stream gives you the correct information as you have seen.

PS M:\Scripts> $exception
Test Error
At line:4 char:9
+         throw "Test Error";
+         ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Test Error:String) [], RuntimeException
    + FullyQualifiedErrorId : Test Error

$exception is an object will properties that contain all of this information so a possible way to get what you need would be to make a custom error string from the desired properties. However that turned out to be more work that it was worth. A simple compromise would be to use Out-String to convert that useful output so that it can be returned as an error.

PS M:\Scripts> throw ($return | out-string)
Test Error
At line:4 char:9
+         throw "Test Error";
+         ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Test Error:String) [], RuntimeException
    + FullyQualifiedErrorId : Test Error

At line:1 char:1
+ throw ($return | out-string)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Test Error
At ...Test Error

:String) [], RuntimeException
    + FullyQualifiedErrorId : Test Error
At line:4 char:9
+         throw "Test Error";
+         ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Test Error:String) [], RuntimeException
    + FullyQualifiedErrorId : Test Error

So now we have a proper terminating error with information from both relative to the scriptblock and to where in the calling code the error is generated. You will see some repetition obviously but maybe this will be a start for you.

If there is other information you need specifically I would delve into the $exception object with Get-Member to see if you can find something specific that you are looking for. Some notable properties here would be

$exception.ErrorCategory_Reason
$exception.PSComputerName
$exception.InvocationInfo.Line
$exception.InvocationInfo.CommandOrigin

The last one would read something like this.

PS M:\Scripts> $exception.InvocationInfo.PositionMessage
At line:4 char:9
+         throw "Test Error";
+         ~~~~~~~~~~~~~~~~~~

Thistledown

Thanks for doing more digging on this - it looks like I'll have to use this approach (stacked exception messages) combined with the serialized information from Frode F.'s answer to create an informative log entry. – Intermit

Answer 3

Hoping someone with more experience can comment on this, my comment seems to have gone overlooked.

I found this article which offers some insight into using Enter-PSSession

Or even better, create a persistent session

$session = New-PSSession localhost
Invoke-Command $session {ping example.com}
Invoke-Command $session {$LASTEXITCODE}

To do debugging using tracing, remember that you need to set the VerbosePreference, DebugPreference etc in the remote session

Invoke-Command $session {$VerbosePreference = ‘continue’}
Invoke-Command $session {Write-Verbose hi}

Nonary

He wants to access the exception that is thrown inside the scriptblock used in Invoke-Command. Using a session vs. computername with the same cmdlet doesn't change the way exceptions are passed to the "management computer". – Lamonica

As it happens, the actual script I'm using does use sessions created with New-PSSession, but as that part is horribly complicated (lots of disconnects & reconnects across processes) and doesn't seem to impact exception handling in this case, I omitted it from the example code. Thanks anyway! – Intermit