Reading input from raw_input() without having the prompt overwritten by other threads in Python

Question

I'm trying to let the user input commands at a console using raw_input(), this works fine. The problem is I have background threads that occasionally output log-information to the screen and when they do they mess up the input prompt (since the output go wherever the cursor happens to be at the moment).

This is a small Python program that illustrate what i mean.

#!/usr/bin/env python
import threading
import time

def message_loop():
    while True:
        time.sleep(1)
        print "Hello World"

thread = threading.Thread(target = message_loop)
thread.start()

while True:
    input = raw_input("Prompt> ")
    print "You typed", input

This is an example of what it could look like when I run it:

Prompt> Hello World
Hello World
Hello World
Hello World
test
You typed test
Prompt> Hello World
Hello World
Hello World
hellHello World
o
You typed hello
Prompt> Hello World
Hello World
Hello World
Hello World

What I want is for the prompt to move along with the output from the thread. Like so:

Hello World
Hello World
Prompt> test
You typed test
Hello World
Hello World
Hello World
Hello World
Hello World
Prompt> hello
You typed hello
Hello World
Hello World
Hello World
Hello World
Prompt> 

Any ideas on how to achieve this without resorting to ugly hacks? :)

Haye

This question is similar to: How to implement a python REPL that nicely handles asynchronous output?. If you believe it’s different, please edit the question, make it clear how it’s different and/or how the answers on that question are not helpful for your problem. – Jenifferjenilee

Answer 1

I recently encountered this problem, and would like to leave this solution here for future reference. These solutions clear the pending raw_input (readline) text from the terminal, print the new text, then reprint to the terminal what was in the raw_input buffer.

This first program is pretty simple, but only works correctly when there is only 1 line of text waiting for raw_input:

#!/usr/bin/python

import time,readline,thread,sys

def noisy_thread():
    while True:
        time.sleep(3)
        sys.stdout.write('\r'+' '*(len(readline.get_line_buffer())+2)+'\r')
        print 'Interrupting text!'
        sys.stdout.write('> ' + readline.get_line_buffer())
        sys.stdout.flush()

thread.start_new_thread(noisy_thread, ())
while True:
    s = raw_input('> ')

Output:

$ ./threads_input.py
Interrupting text!
Interrupting text!
Interrupting text!
> WELL, PRINCE, Genoa and Lucca are now no more than private estates of the Bo
Interrupting text!
> WELL, PRINCE, Genoa and Lucca are now no more than private estates of the Bo
naparte family. No, I warn you, that if you do not tell me we are at war,

The second correctly handles 2 or more buffered lines, but has more (standard) module dependencies and requires a wee bit of terminal hackery:

#!/usr/bin/python

import time,readline,thread
import sys,struct,fcntl,termios

def blank_current_readline():
    # Next line said to be reasonably portable for various Unixes
    (rows,cols) = struct.unpack('hh', fcntl.ioctl(sys.stdout, termios.TIOCGWINSZ,'1234'))

    text_len = len(readline.get_line_buffer())+2

    # ANSI escape sequences (All VT100 except ESC[0G)
    sys.stdout.write('\x1b[2K')                         # Clear current line
    sys.stdout.write('\x1b[1A\x1b[2K'*(text_len/cols))  # Move cursor up and clear line
    sys.stdout.write('\x1b[0G')                         # Move to start of line


def noisy_thread():
    while True:
        time.sleep(3)
        blank_current_readline()
        print 'Interrupting text!'
        sys.stdout.write('> ' + readline.get_line_buffer())
        sys.stdout.flush()          # Needed or text doesn't show until a key is pressed


if __name__ == '__main__':
    thread.start_new_thread(noisy_thread, ())
    while True:
        s = raw_input('> ')

Output. Previous readline lines cleared properly:

$ ./threads_input2.py
Interrupting text!
Interrupting text!
Interrupting text!
Interrupting text!
> WELL, PRINCE, Genoa and Lucca are now no more than private estates of the Bo
naparte family. No, I warn you, that if you do not tell me we are at war,

---

Useful sources:

How to get Linux console window width in Python

apt like column output - python library (This code sample shows how to get terminal width for either Unix or Windows)

http://en.wikipedia.org/wiki/ANSI_escape_code

Perri

blessings module allows to format the output and move around without reaching too deep into terminal's guts. – Skeen

Beware, on some Python versions this breaks if the terminal is resized during runtime due to a bug in the readline module that causes it to ignore terminal resize events (so it doesn't resize its internal buffer accordingly and this breaks the column-counting logic). See bugs.python.org/issue23735. This appears to be fixed in Python 3.5 fortunately. Kick-ass answer otherwise :) – Gratis

Answer 2

I think you need something that lets you dynamically print/delete/overwrite text from the terminal window e.g. how the UNIX watch or top commands work.

I think in your case you would print "Prompt>" but then when you get a "Hello World" you overwrite "Prompt>" with "Hello World", and then print "Prompt>" on the line below. I don't think you can do that with regular output printing to the terminal.

You might be able to do what you want using Python's curses library. I have never used it so I can't tell you how to solve your problem (or if the module will even be able to solve your problem), but I think it is worth taking a look into. A search for "python curses tutorial" provided a PDF tutorial document which seems helpful.

Vibrate

I would rather not have extra dependencies. But since this is only a cosmetic change. I think I'll have a look at curses and just fall back to the current behavior if it doesn't exist :) – Haye

Answer 3

you need to update stdout from a single thread, not from multiple threads... or else you have no control over interleaved i/o.

you will want to create a single thread for output writing.

you can use a Queue in the thread and have all other threads write their output logging info to it.. then read from this Queue and write to stdout at appropriate times along with your prompt message.

Consciousness

The output coming from the other threads are handled in a consistent way is not interleaved with eachother. It's the input that is the problem. Using a queue, and a single thread for both input and output I think I would have to implement my own input handling. raw_input currently blocks the active thread until EOL. – Haye

Answer 4

I don't think it's possible. How should that behave anyways? Nothing shows up until the user presses Enter? If that's so, output would only come when the user issues a command (or whatever your system expects), and that doesn't sound desirable.

Methinks your threads should output to another file.

Semiconscious

No, i want the messages to appear before the user presses enter. But the prompt would just move down to not be overwritten by the messages. – Haye