Java: synchronizing standard out and standard error

Question

I have a strange problem and it would be nice if I could solve it. For the debugging purposes (and some other things, as well) I'm writing a log of a console Java application on the standard output. Some things that are writen on standard out, and some things like errors are printed on standard error. The problem is that these two are not perfectly synchronized, so the order of printed lines is not always correct. I guess this is because many things are printed and it happens that a buffer for one output is full so the other output prints before the first one flushes it buffer.

E.g., I want to write this:

syso: aaa
syso: bbb
syso: ccc
syso: ddd
syso: eee
syserr: ---

What is sometimes printed is

aaa
bbb
ccc
---
ddd
eee

Sometimes there is not a new line in between, so it looks like

aaa
bbb
ccc---

ddd
eee

Every time I print something on an output, I flush the same output with

System.out.flush();

or

System.err.flush();

How to solve this problem? Btw, everything is printed in the Eclipse console.

Augment

Have you considered using a dedicated logging framework like Log4J instead? – Marylyn

@Péter sure, that would be one option, but I would prefer if there was a simple solution to this problem (for future reference, if nothing else). – Augment

Answer 1

public class Util

    synchronized public static void printToOut(...)
        out.print(...)

    synchronized public static void printToErr(...)
        err.print(...)

Autohypnosis

that's possible; even if we sync the two streams in JVM, they can still come out async-ly on terminal devices. – Autohypnosis

Answer 2

The problem is that it's the responsibility of the terminal emulator (in your case, Eclipse) to process the standard output and the standard error of your application. Without communicating with the terminal emulator, you can never be sure that out and err are displayed in the right order. Therefore, I would consider printing everything on err and redirect it to a file. You can still use out for clean user interaction.

Nevertheless, there is a (very bad, but strict) solution to your problem:

System.out.println(...);
System.out.flush();
Thread.sleep(100);

System.err.println(...);
System.err.flush();
Thread.sleep(100);

You may have to change the sleep duration depending on your configuration!

Americanist

Why is this solution very bad? – Gobelin

@skia.heliou Because the delay is completely arbitrary and may vary depending on the speed of the processor etc. – Astra

Answer 3

This is a long-standing Eclipse bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=32205.

Lemuel

The bug has been fixed in Eclipse 2019-09. Now you have the option to synchronize System.out and System.err (disabled by default). However, System.err output will lose its red color and will be black as well. – Xena

Losing the red removes most of the usefulness. – Baillargeon

Answer 4

I know this post is ancient, but it's still an issue today, so here another solution that fixes the problem using @EserAygün's answer, but in a way that does not require you to find and modify every place in your project where you are writing to System.out or System.err.

Create yourself a class called EclipseTools with the following content (and the required package declaration and imports):

public class EclipseTools {

    private static OutputStream lastStream = null;
    private static boolean      isFixed    = false;

    private static class FixedStream extends OutputStream {

        private final OutputStream target;

        public FixedStream(OutputStream originalStream) {
            target = originalStream;
        }

        @Override
        public void write(int b) throws IOException {
            if (lastStream!=this) swap();
            target.write(b);
        }

        @Override
        public void write(byte[] b) throws IOException {
            if (lastStream!=this) swap();
            target.write(b);
        }

        @Override
        public void write(byte[] b, int off, int len) throws IOException {
            if (lastStream!=this) swap();
            target.write(b, off, len);
        }

        private void swap() throws IOException {
            if (lastStream!=null) {
                lastStream.flush();
                try { Thread.sleep(200); } catch (InterruptedException e) {}
            }
            lastStream = this;
        }

        @Override public void close() throws IOException { target.close(); }
        @Override public void flush() throws IOException { target.flush(); }
    }

    /**
     * Inserts a 200ms delay into the System.err or System.out OutputStreams
     * every time the output switches from one to the other. This prevents
     * the Eclipse console from showing the output of the two streams out of
     * order. This function only needs to be called once.
     */
    public static void fixConsole() {
        if (isFixed) return;
        isFixed = true;
        System.setErr(new PrintStream(new FixedStream(System.err)));
        System.setOut(new PrintStream(new FixedStream(System.out)));
    }
}

Then, just call EclipseTools.fixConsole() once in the beginning of your code. Problem solved.

Basically, this replaces the two streams System.err and System.out with a custom set of streams that simply forward their data to the original streams, but keep track of which stream was written to last. If the stream that is written to changes, for example a System.err.something(...) followed by a System.out.something(...), it flushes the output of the last stream and waits for 200ms to give the Eclipse console time to complete printing it.

Note: The 200ms are just a rough initial value. If this code reduces, but does not eliminate the problem for you, increase the delay in Thread.sleep from 200 to something higher until it works. Alternatively, if this delay works but impacts performance of your code (if you alternate streams often), you can try reducing it gradually until you start getting errors.

Lasko

Answer 5

java.lang.System.setErr(java.lang.System.out);

makes the application use the standard output as error stream.

Schoonover

This way I can't make a difference between std out and std err (they are both printed in black and not in black-red as usual). – Augment

Answer 6

The problem lies in the use of the Eclipse Console. Usually, std out will write bytes one at a time to the console, and std err will too, but in red. However, the method does not wait for the bytes to all be written before returning. So, what I recommend is this:

import java.io.OutputStream;
import java.io.PrintStream;
import java.util.function.IntConsumer;

public final class Printer extends PrintStream {
    public static final Printer out = new Printer(
            e -> System.out.print((char) e));
    public static final Printer err = new Printer(
            e -> System.err.print((char) e));
    private final IntConsumer printer;
    private static final Object lock = "";

    private Printer(IntConsumer printer) {
        super(new OutputStream() {
            public void write(int b) {
                printer.accept(b);
            }
        });
        this.printer = printer;
    }

    public void print(int x) {
        synchronized (lock) {
            this.print(Integer.toString(x));
        }
    }

    public void print(boolean x) {
        synchronized (lock) {
            this.print(Boolean.toString(x));
        }
    }

    public void print(double x) {
        synchronized (lock) {
            this.print(Double.toString(x));
        }
    }

    public void print(float x) {
        synchronized (lock) {
            this.print(Float.toString(x));
        }
    }

    public void print(long x) {
        synchronized (lock) {
            this.print(Long.toString(x));
        }
    }

    public void print(char x) {
        synchronized (lock) {
            this.print(Character.toString(x));
        }
    }

    public void print(char[] x) {
        synchronized (lock) {
            StringBuffer str = new StringBuffer(x.length);
            for (char c : x) {
                str.append(c);
            }
            this.print(str);
        }
    }

    public void print(Object x) {
        synchronized (lock) {
            this.print(x.toString());
        }
    }

    public void print(String x) {
        synchronized (lock) {
            x.chars().forEach(printer);
        }
    }

    public void println(int x) {
        synchronized (lock) {
            this.print(Integer.toString(x) + "\n");
        }
    }

    public void println(boolean x) {
        synchronized (lock) {
            this.print(Boolean.toString(x) + "\n");
        }
    }

    public void println(double x) {
        synchronized (lock) {
            this.print(Double.toString(x) + "\n");
        }
    }

    public void println(float x) {
        synchronized (lock) {
            this.print(Float.toString(x) + "\n");
        }
    }

    public void println(long x) {
        this.print(Long.toString(x) + "\n");
    }

    public void println(char x) {
        synchronized (lock) {
            this.print(Character.toString(x) + "\n");
        }
    }

    public void println(char[] x) {
        synchronized (lock) {
            StringBuffer str = new StringBuffer(x.length);
            for (char c : x) {
                str.append(c);
            }
            this.print(str + "\n");
        }
    }

    public void println(Object x) {
        synchronized (lock) {
            this.print(x.toString() + "\n");
        }
    }

    public void println(String x) {
        synchronized (lock) {
            x.chars().forEach(printer);
            printer.accept('\n');
        }
    }
}

Use Printer.out and Printer.err instead of System.out and System.err. It still has the same errors, but this works much better.

Wescott