Powershell: get output from Receive-Job
Asked Answered
P

8

34

I have a collection of jobs that are running. When they complete I use receive-job and it writes the output to the screen. I'd like to take that output and log it to a file. I don't want to tee the output produced while the jobs are running because with multiple jobs running at once the logging would be interspersed. Get-Job | Receive-Job prints the output in a nice organized manner.

I have tried all of the following and no output is written to the file or stored in a variable, it just goes to the screen:

#Wait for last job to complete
While (Get-Job -State "Running") {    
    Log "Running..." -v $info
    Start-Sleep 10        
}    
Log ("Jobs Completed. Output:") -v $info

# Getting the information back from the jobs
foreach($job in Get-Job){
    Receive-Job -Job $job | Out-File c:\Test.log -Append
    $output = Receive-Job -Job $job        
    Log ("OUTPUT: "+$output)
    Receive-Job -Job $job -OutVariable $foo
    Log ("FOO: "+$foo)
}

EDIT: I have removed the extra Receive-Job calls in the foreach to the following after seeing Keith's comment:

# Getting the information back from the jobs
foreach($job in Get-Job){
    Receive-Job -Job $job -OutVariable temp
    Write-Host ("Temp: "+$temp)
    $temp | Out-File -FilePath c:\Test.log -Append 
}

I also verified I'm not using Receive-Job anywhere else in the script. The write-host $temp and the out-file still produce no output though.

Pizarro answered 15/8, 2012 at 17:6 Comment(2)
This is a great question. As you mention below, Write-Output is not helpful since it interferes with what is returned by a function call. I thought Write-Verbose might prove helpful as you could do ReadAll on the job's child job's Verbose field, but it only is populated if the VerbosePreference is set, which means, it shows in the output too. Really I think we need this bug fixed: github.com/PowerShell/PowerShell/issues/3130Tithing
See @Spongebob the codder answer below. It lets you easily get the output of your child job's Write-Host into a variable, which is great. I use this in combination with Start-Transcript/Stop-Transcript in my jobs. Then I've got per-job logs, but the logs of each job available to inspect with each Receive-Job. There's still the issue of your top-level console getting polluted with the Write-Host calls of the child jobs, but no solution I've seen here gets around that.Tithing
P
13

NOTE: After some experimentation I have found that using write-output or just outputting a variable directly is not a good solution either. If you have a function that uses either of those methods and also returns a value the output will be concatenated with the return value!

The best way I have found to log output from jobs is to use write-host in combination with the Start-Transcript and Stop-Transcript cmdlets. There are some disadvantages such as the ISE does not support Transcript cmdlets. Also I haven't found a way to output to the transcript without it going to the host (I wanted a robust log with a less verbose user experience), but it seems to be the best solution available currently. It is the only way I've been able to log concurrent jobs without interspersing the output without having to right multiple temporary log files for each job and then combining them.

Below was my previous answer that I left just for documentation reasons:

The issue here was not getting output from Receive-Job. The output from the jobs I was expecting to see is being written in the job by write-host. When receive-job was called I saw all my output on the screen, but nothing was available to pipe elsewhere. It seems when I call receive-job all those write-host cmdlets execute, but there's nothing for me to pick up and pipe to out-file. The following script demonstrates this. You'll notice "Sleepy Time" appears in both the log file and the host, while "Slept" only appears in the host. So, rather than trying to pipe Receive-Job to write-host, I need to modify my Log function to not use write-host or to split the output as shown below.

cls
rm c:\jobs.log
$foo = @(1,2,3)
$jobs = @()

foreach($num in $foo){
    $s = { get-process -name "explorer"
           Start-Sleep $args[0]
           $x = ("Sleepy Time: "+$args[0])
           write-host $x
           $x
           write-host ("Slept: "+$args[0])
         }

    $id = [System.Guid]::NewGuid()
    $jobs += $id   
    Start-Job -Name $id -ScriptBlock $s -args $num
}

While (Get-Job -State "Running") {
    cls
    Get-Job
    Start-Sleep 1 
}
cls
Get-Job
write-host "Jobs completed, getting output"

foreach($job in $jobs){
    Receive-Job -Name $job | out-file c:/jobs.log -append    
}

Remove-Job *
notepad c:\jobs.log

Below is a sample that will run concurrent jobs and then log them sequentially as per mbourgon's request. You'll notice the timestamps indicate the work was interspersed, but the logging is displayed sequentially by job.

#This is an example of logging concurrent jobs sequentially
#It must be run from the powershell prompt as ISE does not support transcripts

cls
$logPath = "c:\jobs.log"
rm $logPath

Start-Transcript -Path $logPath -Append

#Define some stuff
$foo = @(0,1,2)
$bar = @(@(1,"AAA"),@(2,"BBB"),@(3,"CCC"))

$func = {
    function DoWork1 {
        Log "This is DoWork1"
        return 0
    }

    function DoWork2 {
        Log "This is DoWork2"
        return 0
    }

    function Log {
        Param([parameter(ValueFromPipeline=$true)]$InputObject)
        $nl = [Environment]::NewLine.Chars(0)
        $time = Get-Date -Format {HH:mm:ss} 
        Write-Host ($time+">"+$InputObject+$nl)
    }
}

#Start here
foreach($num in $foo){
    $s = { $num = $args[0]
           $bar = $args[1]           
           $logPath = $args[2]
           Log ("Num: "+$num) 

           for($i=0; $i -lt 5; $i++){
                $out = ([string]::$i+$bar[$num][1])
                Log $out
                start-sleep 1
           }
           Start-Sleep $num

           $x = DoWork1
           Log ("The value of x is: "+$x)           

           $y = DoWork2               
           Log ("The value of y is: "+$y)               
         }
    Start-Job -InitializationScript $func -ScriptBlock $s -args $num, $bar, $logPath
}

While (Get-Job -State "Running") {
    cls
    Get-Job
    Start-Sleep 1 
}
cls
Get-Job
write-host "Jobs completed, getting output"    

Get-Job | Receive-Job

Remove-Job *
Stop-Transcript
notepad $logPath
Pizarro answered 16/8, 2012 at 16:41 Comment(3)
is there any way to get a copy of the "current best way" you mention? Thanks; I'm having the same problem.Disjunctive
I have added a code example to the end of my answer, hope this is useful.Pizarro
I posted this as a request to the Powershell team github.com/PowerShell/PowerShell/issues/17482Winfield
R
21

If the job uses Write-Host to produce output, Receive-Job returns $null, but the results get written to the host. However, if the job uses Write-Output to produce output in lieu of Write-Host, Receive-Job returns a string array [string[]] of the job output.

To demonstrate, enter this innocuous code at the PowerShell prompt:

$job = Start-Job -ScriptBlock {
    [int] $counter = 0
    while ($counter -lt 10) {
        Write-Output "Counter = $counter."
        Start-Sleep -Seconds 5
        $counter++
    }
}

Wait about 20-30 seconds for the job to produce some output, then enter this code:

$result = Receive-Job -Job $job
$result.Count
$result
$result | Get-Member

The $result object contains the strings produced by the job.

Rothman answered 28/1, 2015 at 19:41 Comment(0)
P
13

NOTE: After some experimentation I have found that using write-output or just outputting a variable directly is not a good solution either. If you have a function that uses either of those methods and also returns a value the output will be concatenated with the return value!

The best way I have found to log output from jobs is to use write-host in combination with the Start-Transcript and Stop-Transcript cmdlets. There are some disadvantages such as the ISE does not support Transcript cmdlets. Also I haven't found a way to output to the transcript without it going to the host (I wanted a robust log with a less verbose user experience), but it seems to be the best solution available currently. It is the only way I've been able to log concurrent jobs without interspersing the output without having to right multiple temporary log files for each job and then combining them.

Below was my previous answer that I left just for documentation reasons:

The issue here was not getting output from Receive-Job. The output from the jobs I was expecting to see is being written in the job by write-host. When receive-job was called I saw all my output on the screen, but nothing was available to pipe elsewhere. It seems when I call receive-job all those write-host cmdlets execute, but there's nothing for me to pick up and pipe to out-file. The following script demonstrates this. You'll notice "Sleepy Time" appears in both the log file and the host, while "Slept" only appears in the host. So, rather than trying to pipe Receive-Job to write-host, I need to modify my Log function to not use write-host or to split the output as shown below.

cls
rm c:\jobs.log
$foo = @(1,2,3)
$jobs = @()

foreach($num in $foo){
    $s = { get-process -name "explorer"
           Start-Sleep $args[0]
           $x = ("Sleepy Time: "+$args[0])
           write-host $x
           $x
           write-host ("Slept: "+$args[0])
         }

    $id = [System.Guid]::NewGuid()
    $jobs += $id   
    Start-Job -Name $id -ScriptBlock $s -args $num
}

While (Get-Job -State "Running") {
    cls
    Get-Job
    Start-Sleep 1 
}
cls
Get-Job
write-host "Jobs completed, getting output"

foreach($job in $jobs){
    Receive-Job -Name $job | out-file c:/jobs.log -append    
}

Remove-Job *
notepad c:\jobs.log

Below is a sample that will run concurrent jobs and then log them sequentially as per mbourgon's request. You'll notice the timestamps indicate the work was interspersed, but the logging is displayed sequentially by job.

#This is an example of logging concurrent jobs sequentially
#It must be run from the powershell prompt as ISE does not support transcripts

cls
$logPath = "c:\jobs.log"
rm $logPath

Start-Transcript -Path $logPath -Append

#Define some stuff
$foo = @(0,1,2)
$bar = @(@(1,"AAA"),@(2,"BBB"),@(3,"CCC"))

$func = {
    function DoWork1 {
        Log "This is DoWork1"
        return 0
    }

    function DoWork2 {
        Log "This is DoWork2"
        return 0
    }

    function Log {
        Param([parameter(ValueFromPipeline=$true)]$InputObject)
        $nl = [Environment]::NewLine.Chars(0)
        $time = Get-Date -Format {HH:mm:ss} 
        Write-Host ($time+">"+$InputObject+$nl)
    }
}

#Start here
foreach($num in $foo){
    $s = { $num = $args[0]
           $bar = $args[1]           
           $logPath = $args[2]
           Log ("Num: "+$num) 

           for($i=0; $i -lt 5; $i++){
                $out = ([string]::$i+$bar[$num][1])
                Log $out
                start-sleep 1
           }
           Start-Sleep $num

           $x = DoWork1
           Log ("The value of x is: "+$x)           

           $y = DoWork2               
           Log ("The value of y is: "+$y)               
         }
    Start-Job -InitializationScript $func -ScriptBlock $s -args $num, $bar, $logPath
}

While (Get-Job -State "Running") {
    cls
    Get-Job
    Start-Sleep 1 
}
cls
Get-Job
write-host "Jobs completed, getting output"    

Get-Job | Receive-Job

Remove-Job *
Stop-Transcript
notepad $logPath
Pizarro answered 16/8, 2012 at 16:41 Comment(3)
is there any way to get a copy of the "current best way" you mention? Thanks; I'm having the same problem.Disjunctive
I have added a code example to the end of my answer, hope this is useful.Pizarro
I posted this as a request to the Powershell team github.com/PowerShell/PowerShell/issues/17482Winfield
E
11

I found soulution how to receive Write-Host results to variable with Receive-Job CMDlet, it's working on Powershell 5.1:

$var = Receive-Job $job 6>&1
Efrenefron answered 16/2, 2017 at 5:30 Comment(2)
if someone is curious how it works:learn.microsoft.com/en-us/powershell/module/… from msdn: >&1 Redirects the specified stream to the Success stream.Momentum
But is it certain that the file number 6 is the correct one, or is this just based on observation and in no way certain to always be the case?Luci
L
6

I know this is old. I don't use jobs very often so I was obviously having the same problem when I stumbled on this thread. Anyway, I arrived at a different solution so I thought I'd throw it out there.

Also in PowerShell 5.1 I simply referenced the properties on the Job Object(s) In my case this looked like:

$Jobs.ChildJobs.Output | Out-File <LogFile> -Append

It looks like all the streams are recorded in this way. Simple echoes go to the Output, presumable the success stream. Write-Host commands go to Information Stream.

Output        : {}
Error         : {}
Progress      : {}
Verbose       : {}
Debug         : {}
Warning       : {}
Information   : {}
Lupine answered 12/2, 2020 at 1:57 Comment(0)
L
4

When you receive the results of a job those results are deleted such that further calls to Receive-Job will have nothing to retrieve (assuming the job is completed). If you don't want Receive-Job to delete the results use the -Keep switch. Now this doesn't explain why your first call to Receive-Job isn't outputting anything to the log file ... unless there is a part of the script you're not showing that is doing a Receive-Job before this.

BTW when you use -OutVariable supply it what a variable name and not a variable e.g. -ov temp. Then you later access that info with $temp.

Lota answered 15/8, 2012 at 17:17 Comment(3)
Keith I modified my script so receive-job is only called once and verified receive-job is not used anywhere else in the script.Pizarro
BTW when you use -OutVariable supply it what a variable name and not a variable e.g. -ov temp. Then you later access that info with $temp.Lota
Yup, sorry that was a typo... I've been changing this thing quite a bit.Pizarro
F
3

You could pipe your Get-Process to Format-Table, so you can see all of the output.

    Get-Process -Name "explorer" | format-Table -hidetableheaders -AutoSize

And you could use something like this at the end of the script...

    Get-job | Receive-Job 2>&1 >> c:\path\to\your\log\file.log

The "2>&1 >>" with grab all output from Get-Job | Receive-Job

Fun answered 20/12, 2013 at 1:40 Comment(0)
D
1

A simple solution to the problem. Write-Verbose "blabla" -Verbose


$s = New-PSSession -ComputerName "Computer"
$j = Invoke-Command -Session $s  -ScriptBlock { Write-Verbose "Message" -Verbose } -AsJob
Wait-Job $j

$Child = $j.ChildJobs[0]
$Child.Verbose | out-file d:/job.log -append
Diplomat answered 9/11, 2016 at 14:18 Comment(0)
P
1

What I have done is to not call Receive-Job at all. But create a wrapper scriptblock that starts and stops a transcripts. Then in that wrapper script block I call the jobs script block like

&$ScriptBlock < Parameters >

So the transcript will capture all the output to a file.

Pincenez answered 19/6, 2021 at 21:52 Comment(1)
Could you edit with a more clear example and then tag me back please? I'd like to try this and will +1 if you can give some clarity on example and syntax.Truckle

© 2022 - 2024 — McMap. All rights reserved.