Handle large amounts of output from p.WaitForExit() with RedirectStandardOutput = True

Question

As discussed here ProcessStartInfo hanging on "WaitForExit"? Why? - calling p.WaitForExit() with a large output fills the OutputStream and causes a deadlock as the process and the output stream wait for each other.

My code for example:

Dim p = New Process()
Dim ReturnValue As Boolean = False
p.StartInfo = New ProcessStartInfo(LynxPath, "-dump -nolist -width 1000 " & HtmlBuffer)
p.StartInfo.WorkingDirectory = WorkingRoot
p.StartInfo.UseShellExecute = False
p.StartInfo.RedirectStandardOutput = True
p.StartInfo.RedirectStandardError = True
p.StartInfo.WindowStyle = ProcessWindowStyle.Hidden
p.StartInfo.CreateNoWindow = True
p.Start()
ReturnValue = p.WaitForExit(5000)

When dealing with a large output from LYNX the thread hangs unless i use a timeout as above, and the output is trimmed when the output buffer gets full - meaning any output i do read isnt complete.

The c# solution posted in the question above seems to get around this by making use of the OutputDataReceived event on the process class. My problem however is converting that c# code into vb.net 3.5, i ran it through the normal conversion routes and it spat out:

Using process As New Process()
    process.StartInfo.FileName = filename
    process.StartInfo.Arguments = arguments
    process.StartInfo.UseShellExecute = False
    process.StartInfo.RedirectStandardOutput = True
    process.StartInfo.RedirectStandardError = True

    Dim output As New StringBuilder()
    Dim [error] As New StringBuilder()

    Using outputWaitHandle As New AutoResetEvent(False)
        Using errorWaitHandle As New AutoResetEvent(False)
            process.OutputDataReceived += Function(sender, e) 
            If e.Data Is Nothing Then
                outputWaitHandle.[Set]()
            Else
                output.AppendLine(e.Data)
            End If

End Function
            process.ErrorDataReceived += Function(sender, e) 
            If e.Data Is Nothing Then
                errorWaitHandle.[Set]()
            Else
                [error].AppendLine(e.Data)
            End If

End Function

            process.Start()

            process.BeginOutputReadLine()
            process.BeginErrorReadLine()

                ' Process completed. Check process.ExitCode here.
            If process.WaitForExit(timeout) AndAlso outputWaitHandle.WaitOne(timeout) AndAlso errorWaitHandle.WaitOne(timeout) Then
                ' Timed out.
            Else
            End If
        End Using
    End Using
End Using

However Visual Studio 2008 flags the function declerations as invalid syntax (Inline methods such as this are in 4.5 onwards i think?), how would one make use of this example in 3.5 - cant quite get my head round it.

Edit: just found this link: http://msdn.microsoft.com/en-us/library/system.diagnostics.process.outputdatareceived.aspx - Trying to figure it out now

Twelvetone

You might be interested in this post, which explains how you can handle process streams using async tasks instead of the clunkier data received events – Colligan

Answer 1

This is what we ultimitly came up with, it takes a html string, writes it to a file then opens it with lynx, The lynx output is then captured using an event handle (which handles large amounts of output) and populates a plain text string:

Function ConvertHtmlToPlainText(ByVal HtmlString As String) As String

    ' Define FileBuffer Path
    Dim HtmlBuffer As String = WorkingRoot & "HtmlBuffer.html"

    ' Delete any old buffer files
    Try
        If File.Exists(HtmlBuffer) = True Then
            File.Delete(HtmlBuffer)
        End If
    Catch ex As Exception
        Return "Error: Deleting old buffer file: " & ex.Message
    End Try

    ' Write the HTML to the buffer file
    Try
        File.WriteAllText(WorkingRoot & "HtmlBuffer.html", HtmlString)
    Catch ex As Exception
        Return "Error: Writing new buffer file: " & ex.Message
    End Try

    ' Check the file was written OK
    If File.Exists(HtmlBuffer) = False Then
        Return "Error: HTML Buffer file was not written successfully."
    End If

    If File.Exists(LynxPath) = False Then
        Return "Error: Lynx.exe could not be found in path: " & LynxPath
    End If

    ' Read the buffer file with Lynx and capture plain text output
    Try
        LynxOutput = ""
        LynxOutputLineCount = 0
        Dim LynxProcess As New Process()
        LynxProcess.StartInfo = New ProcessStartInfo(LynxPath, "-dump -nolist -width 1000 " & HtmlBuffer)
        LynxProcess.StartInfo.UseShellExecute = False
        LynxProcess.StartInfo.RedirectStandardOutput = True
        LynxProcess.StartInfo.WindowStyle = ProcessWindowStyle.Hidden
        LynxProcess.StartInfo.CreateNoWindow = True
        AddHandler LynxProcess.OutputDataReceived, AddressOf LynxOutputHandle
        LynxProcess.StartInfo.RedirectStandardInput = True
        LynxProcess.Start()
        LynxProcess.BeginOutputReadLine()
        LynxProcess.WaitForExit()
        LynxProcess.Close()

        ' Return the rendered Text
        Return TrimEachLine(LynxOutput.Replace(vbLf & vbLf, vbLf))

    Catch ex As Exception
        Return "Error: Error running LYNX to parse the buffer: " & ex.Message
    End Try
End Function

Private Sub LynxOutputHandle(ByVal sendingProcess As Object, ByVal outLine As DataReceivedEventArgs)
    LynxOutputLineCount = LynxOutputLineCount + 1
    LynxOutput = LynxOutput & outLine.Data & vbLf
End Sub

Twelvetone