sys.stdin.readlines() hangs Python script

Question

Everytime I'm executing my Python script, it appears to hang on this line:

lines = sys.stdin.readlines()

What should I do to fix/avoid this?

EDIT

Here's what I'm doing with lines:

lines = sys.stdin.readlines()
updates = [line.split() for line in lines]

EDIT 2

I'm running this script from a git hook so is there anyway around the EOF?

Merat

Do note that readlines() requires an EOF before it will return. This won't happen until the an EOF is given to the stdin, via the executing application/shell. – Mechanistic

Answer 1

This depends a lot on what you are trying to accomplish. You might be able do:

for line in sys.stdin:
    #do something with line

Of course, with this idiom as well as the readlines() method you are using, you need to somehow send the EOF character to your script so that it knows that the file is ready to read. (On unix Ctrl-D usually does the trick).

Dornick

I'm running this script from a git hook so is there anyway around the EOF? – Merat

@BoA -- Sorry, I don't know anything about git hooks, although it seems that if your program is reading from a pipe, it should just work. – Dornick

@Dornick Basically it just runs the python script automatically after a commit. – Merat

@BoA -- That makes sense, but why can't you hit Ctrl-D? Where is your script getting input from? – Dornick

@Dornick The script is run automatically, I don't run it manually. Do I still need to press Ctrl+D? How do I know when to press it? – Merat

@BoA -- you press it when you're done typing the stuff you want to send to your program... – Dornick

@Dornick Ah, sorry was thinking about something else, you're right. – Merat

5-minute edit limit bit me again. Had to step away. Yea I helps if I actually take a moment to read through the entire answer. :) – Mechanistic

Answer 2

Unless you are redirecting something to stdin that would be expected behavior. That says to read input from stdin (which would be the console you are running the script from). It is waiting for your input.

See: "How to finish sys.stdin.readlines() input?

Psychotherapy

Answer 3

If you're running the program in an interactive session, then this line causes Python to read from standard input (i. e. your keyboard) until you send the EOF character (Ctrl-D (Unix/Mac) or Ctrl-Z (Windows)).

>>> import sys
>>> a = sys.stdin.readlines()
Test
Test2
^Z
>>> a
['Test\n', 'Test2\n']

Dealt

Answer 4

I know this isn't directly answering your question, as others have already addressed the EOF issue, but typically what I've found that works best when reading live output from a long lived subprocess or stdin is the while/if line approach:

while True:
    line = sys.stdin.readline()
    if not line:
       break
    process(line)

In this case, sys.stdin.readline() will return lines of text before an EOF is returned. Once the EOF if given, the empty line will be returned which triggers the break from the loop. A hang can still occur here, as long as an EOF isn't provided.

It's worth noting that the ability to process the "live output", while the subprocess/stdin is still running, requires the writing application to flush it's output.

Mechanistic

I think @mgilson's answer would work better; this would work, but it's the more "Pythonic" way – Hydroponics

I completely agree with the statement that @mgilson's answer is more pythonic. I just haven't found it to be very friendly for "live output", such as capturing status information from a long running application to report status back to the user, as it requires the file like object to produce an EOF before the loop starts processing. In my example, the call to readline() will give me lines of text before an EOF is returned. The EOF is only required to break from the while, by giving me an empty line. That being said, I probably could improve my description because that isn't very clear. – Mechanistic

Curious that it works for you, because I can show the aforementioned behavior on at least up to Python 2.6.6 for both sys.stdin and subprocess objects and I'm not the only one who has found this to be the case: #2805043. Don't get me wrong, I am more than welcome to accept that I may be wrong because I would much prefer the simpler syntax. – Mechanistic

I did just notice though, that the following comment was added to the question above: for file.readline() vs. for line in file see bugs.python.org/issue3907 (in short: it works on Python3; use io.open() on Python 2.6+) – Mechanistic