C# - Realtime console output redirection

Question

I'm developing a C# application and I need to start an external console program to perform some tasks (extract files). What I need to do is to redirect the output of the console program. Code like this one does not work, because it raises events only when a new line is writen in the console program, but the one I use "updates" what's shown in the console window, without writting any new lines. How can I raise an event every time the text in the console is updated? Or just get the output of the console program every X seconds? Thanks in advance!

Clairvoyant

Answer 1

I have had a very similar (possibly the exact) problem as you describe:

1. I needed the console updates to be delivered to me asynchronously.
2. I needed the updates to be detected regardless of whether a newline was input.

What I ended up doing goes like this:

1. Start an "endless" loop of calling StandardOutput.BaseStream.BeginRead.
2. In the callback for BeginRead, check if the return value of EndRead is 0; this means that the console process has closed its output stream (i.e. will never write anything to standard output again).
3. Since BeginRead forces you to use a constant-length buffer, check if the return value of EndRead is equal to the buffer size. This means that there may be more output waiting to be read, and it may be desirable (or even necessary) that this output is processed all in one piece. What I did was keep a StringBuilder around and append the output read so far. Whenever output is read but its length is < the buffer length, notify yourself (I do it with an event) that there is output, send the contents of the StringBuilder to the subscriber, and then clear it.

However, in my case I was simply writing more stuff to the console's standard output. I 'm not sure what "updating" the output means in your case.

Update: I just realized (isn't explaining what you are doing a great learning experience?) that the logic outlined above has an off-by-one bug: If the length of the output read by BeginRead is exactly equal to the length of your buffer, then this logic will store the output in the StringBuilder and block while trying to see if there's more output to append. The "current" output will only be sent back to you when/if more output is available, as part of a larger string.

Obviously some method of guarding against this (or a biggish buffer plus faith in your powers of luck) is needed to do this 100% correctly.

Update 2 (code):

DISCLAIMER: This code is not production-ready. It is the result of me quickly hacking together a proof of concept solution to do what needed to be done. Please do not use it as it stands in your production application. If this code causes horrible things to happen to you, I will pretend someone else wrote it.

public class ConsoleInputReadEventArgs : EventArgs
{
    public ConsoleInputReadEventArgs(string input)
    {
        this.Input = input;
    }

    public string Input { get; private set; }
}

public interface IConsoleAutomator
{
    StreamWriter StandardInput { get; }

    event EventHandler<ConsoleInputReadEventArgs> StandardInputRead;
}

public abstract class ConsoleAutomatorBase : IConsoleAutomator
{
    protected readonly StringBuilder inputAccumulator = new StringBuilder();

    protected readonly byte[] buffer = new byte[256];

    protected volatile bool stopAutomation;

    public StreamWriter StandardInput { get; protected set; }

    protected StreamReader StandardOutput { get; set; }

    protected StreamReader StandardError { get; set; }

    public event EventHandler<ConsoleInputReadEventArgs> StandardInputRead;

    protected void BeginReadAsync()
    {
        if (!this.stopAutomation) {
            this.StandardOutput.BaseStream.BeginRead(this.buffer, 0, this.buffer.Length, this.ReadHappened, null);
        }
    }

    protected virtual void OnAutomationStopped()
    {
        this.stopAutomation = true;
        this.StandardOutput.DiscardBufferedData();
    }

    private void ReadHappened(IAsyncResult asyncResult)
    {
        var bytesRead = this.StandardOutput.BaseStream.EndRead(asyncResult);
        if (bytesRead == 0) {
            this.OnAutomationStopped();
            return;
        }

        var input = this.StandardOutput.CurrentEncoding.GetString(this.buffer, 0, bytesRead);
        this.inputAccumulator.Append(input);

        if (bytesRead < this.buffer.Length) {
            this.OnInputRead(this.inputAccumulator.ToString());
        }

        this.BeginReadAsync();
    }

    private void OnInputRead(string input)
    {
        var handler = this.StandardInputRead;
        if (handler == null) {
            return;
        }

        handler(this, new ConsoleInputReadEventArgs(input));
        this.inputAccumulator.Clear();
    }
}

public class ConsoleAutomator : ConsoleAutomatorBase, IConsoleAutomator
{
    public ConsoleAutomator(StreamWriter standardInput, StreamReader standardOutput)
    {
        this.StandardInput = standardInput;
        this.StandardOutput = standardOutput;
    }

    public void StartAutomate()
    {
        this.stopAutomation = false;
        this.BeginReadAsync();
    }

    public void StopAutomation()
    {
        this.OnAutomationStopped();
    }
}

Used like so:

var processStartInfo = new ProcessStartInfo
    {
        FileName = "myprocess.exe",
        RedirectStandardInput = true,
        RedirectStandardOutput = true,
        UseShellExecute = false,
    };

var process = Process.Start(processStartInfo);
var automator = new ConsoleAutomator(process.StandardInput, process.StandardOutput);

// AutomatorStandardInputRead is your event handler
automator.StandardInputRead += AutomatorStandardInputRead;
automator.StartAutomate();

// do whatever you want while that process is running
process.WaitForExit();
automator.StandardInputRead -= AutomatorStandardInputRead;
process.Close();

Abhorrence

By updating I mean this: The console program has a line saying: "Extracting file: 1". Then the number becomes 2, 3, 4, etc (without writting any new lines). Could you also post your code example? (I'm not an expert and I might get confused). – Clairvoyant

Thank you very much! I'll try it as soon as possible. – Clairvoyant

Answer 2

Or alternatively, according to the keep it sane principle, you could read the documentation and do it properly:

var startinfo = new ProcessStartInfo(@".\consoleapp.exe")
{
    CreateNoWindow = true,
    UseShellExecute = false,
    RedirectStandardOutput = true,
    RedirectStandardError = true,
};

var process = new Process { StartInfo = startinfo };
process.Start();

var reader = process.StandardOutput;
while (!reader.EndOfStream)
{
    // the point is that the stream does not end until the process has 
    // finished all of its output.
    var nextLine = reader.ReadLine();
}

process.WaitForExit();

Fellers

The problem is the ReadLine(), ReadToend(), Peek() and whatever you try to use locks and do not return as described in the documentation. May work on file streams and memory streams but not so great when you attach to a stream reader of a console application. If you want to capture raw data from a console it is nice to get all linebreaks. – Kinesiology

You loose the new line charachters! – Streetwalker

It is even worse: ReadLine() blocks sometimes and sometimes it returns null if there is no data. Holy crap! – Lute

It's supposed to block! – Fellers

StandardOutput.EndOfStream blocks. – Loden

Answer 3

According to keep it simple principle I'm posting more compact code.

In my opinion Read is enough in this case.

    private delegate void DataRead(string data);
    private static event DataRead OnDataRead;

    static void Main(string[] args)
    {
        OnDataRead += data => Console.WriteLine(data != null ? data : "Program finished");
        Thread readingThread = new Thread(Read);
        ProcessStartInfo info = new ProcessStartInfo()
        {
            FileName = Environment.GetCommandLineArgs()[0],
            Arguments = "/arg1 arg2",
            RedirectStandardOutput = true,
            UseShellExecute = false,
        };
        using (Process process = Process.Start(info))
        {
            readingThread.Start(process);
            process.WaitForExit();
        }
        readingThread.Join();
    }

    private static void Read(object parameter)
    {
        Process process = parameter as Process;
        char[] buffer = new char[Console.BufferWidth];
        int read = 1;
        while (read > 0)
        {
            read = process.StandardOutput.Read(buffer, 0, buffer.Length);
            string data = read > 0 ? new string(buffer, 0, read) : null;
            if (OnDataRead != null) OnDataRead(data);
        }
    }

Points of interest:

• changing read buffer size
• making a nice class
• making nicer event
• starting process in another thread (so that ui thread is not blocked with Process.WaitForExit)

Goodoh

Answer 4

The struggle is over

Thanks to the above samples, I solved the problems with the StandardOutput and StandardError stream readers blocking and impossible to use directly.

MS admits here about locking problems: system.io.stream.beginread

Subscribing to StandardOutput and StandardError events using process.BeginOutputReadLine() and process.BeginErrorReadLine() and subscribing to OutputDataReceived and ErrorDataReceived works fine but I miss out on newline characters and can not emulate what is happening on the original console being listened to.

This class takes a reference to the StreamReader but captures console output from the StreamReader.BaseStream. The DataReceived event will deliver stream data forever as it arrives. It is not blocking when tested on a foreign console app.

/// <summary>
/// Stream reader for StandardOutput and StandardError stream readers
/// Runs an eternal BeginRead loop on the underlaying stream bypassing the stream reader.
/// 
/// The TextReceived sends data received on the stream in non delimited chunks. Event subscriber can
/// then split on newline characters etc as desired.
/// </summary>
class AsyncStreamReader
{ 

    public delegate void EventHandler<args>(object sender, string Data);
    public event EventHandler<string> DataReceived;

    protected readonly byte[] buffer = new byte[4096];
    private StreamReader reader;


    /// <summary>
    ///  If AsyncStreamReader is active
    /// </summary>
    public bool Active { get; private set; }

    public void Start()
    {
        if (!Active)
        {
            Active = true;
            BeginReadAsync();
        }           
    }

    public void Stop()
    {
        Active=false;         
    }

    public AsyncStreamReader(StreamReader readerToBypass)
    {
        this.reader = readerToBypass;
        this.Active = false;
    }

    protected void BeginReadAsync()
    {
        if (this.Active)
        {
            reader.BaseStream.BeginRead(this.buffer, 0, this.buffer.Length, new AsyncCallback(ReadCallback), null);
        }
    }

    private void ReadCallback(IAsyncResult asyncResult)
    {
        var bytesRead = reader.BaseStream.EndRead(asyncResult);

        string data = null;

        //Terminate async processing if callback has no bytes
        if (bytesRead > 0)
        {
            data = reader.CurrentEncoding.GetString(this.buffer, 0, bytesRead);
        }
        else
        {
            //callback without data - stop async
            this.Active = false;                
        }

        //Send data to event subscriber - null if no longer active
        if (this.DataReceived != null)
        {
            this.DataReceived.Invoke(this, data);
        }

        //Wait for more data from stream
        this.BeginReadAsync();
    }

}

Maybe an explicit event when the AsyncCallback is exciting instead of sending a null string would be nice but the basic problem was solved.

The 4096-size buffer could be smaller. The callback will just loop until all data is delivered.

Use like this:

standardOutput = new AsyncStreamReader(process.StandardOutput);
standardError = new AsyncStreamReader(process.StandardError);

standardOutput.DataReceived += (sender, data) =>
{
    //Code here
};

standardError.DataReceived += (sender, data) =>
{
    //Code here
};

StandardOutput.Start();
StandardError.Start();

Kinesiology

Would you please explain more clearly what you mean with "but I miss out on newline characters" ??? – Lute

Have a look at what I posted. Using binary makes it easier. Right? – Chrissa

Answer 5

Jon said "I'm not sure what "updating" the output means in your case" and I don't know what it means for him either. So I wrote a program that can be used for redirection of its output so we can clearly define the requirements.

It is possible to move the cursor in a console using the Console.CursorLeft Property. However when I used that I was unable to redirect the output, I got an error; something about an invalid stream I think. So then I tried backspace characters, as has already been suggested. So the program I am using to redirect the out from is the following.

class Program
{
    static readonly string[] Days = new [] {"Monday", "Tuesday", "Wednesday",
        "Thursday", "Friday", "Saturday", "Sunday"};
    static int lastlength = 0;
    static int pos = 0;

    static void Main(string[] args)
    {
        Console.Write("Status: ");
        pos = Console.CursorLeft;
        foreach (string Day in Days)
        {
            Update(Day);
        }
        Console.WriteLine("\r\nDone");
    }

    private static void Update(string day)
    {
        lastlength = Console.CursorLeft - pos;
        Console.Write(new string((char)8, lastlength));
        Console.Write(day.PadRight(lastlength));
        Thread.Sleep(1000);
    }
}

When I use the accepted answer to redirect the output of that it works.

I was using some sample code for something entirely different and it was able to process standard output as soon as it is available as in this question. It reads standard output as binary data. So I tried that and the following is an alternative solution for here.

class Program
{
    static Stream BinaryStdOut = null;

    static void Main(string[] args)
    {
        const string TheProgram = @" ... ";
        ProcessStartInfo info = new ProcessStartInfo(TheProgram);
        info.RedirectStandardOutput = true;
        info.UseShellExecute = false;
        Process p = Process.Start(info);
        Console.WriteLine($"Started process {p.Id} {p.ProcessName}");
        BinaryStdOut = p.StandardOutput.BaseStream;
        string Message = null;
        while ((Message = GetMessage()) != null)
            Console.WriteLine(Message);
        p.WaitForExit();
        Console.WriteLine("Done");
    }

    static string GetMessage()
    {
        byte[] Buffer = new byte[80];
        int sizeread = BinaryStdOut.Read(Buffer, 0, Buffer.Length);
        if (sizeread == 0)
            return null;
        return Encoding.UTF8.GetString(Buffer);
    }
}

Actually, this might not be any better than the answer from marchewek but I suppose I will leave this here anyway.

Chrissa