Python subprocess.Popen PIPE and SIGPIPE

Question

While I browsed posts, I ran into this example below on here, It is saying proc1.stdout.close() is needed to be called for appropriate exit of proc1, generating SIGPIPE.

import subprocess

proc1 = subprocess.Popen(['ps', 'cax'], stdout=subprocess.PIPE)
proc2 = subprocess.Popen(['grep', 'python'], stdin=proc1.stdout,
                         stdout=subprocess.PIPE, stderr=subprocess.PIPE)

proc1.stdout.close() # Allow proc1 to receive a SIGPIPE if proc2 exits.
out, err = proc2.communicate()
print('out: {0}'.format(out))
print('err: {0}'.format(err))

However, I am not clear on that. Please fix my understanding.

1. SIGPIPE occurs when a PIPE tries to write to closed PIPE.
2. Writer PIPE is proc1's stdout and reader PIPE is proc2's stdin.
3. proc1 will exit when proc2 exit and proc1 tries to write data to proc2's stdin PIPE. because
   • proc2's stdin PIPE is closed when proc2 exit
   • SIGPIPE happen at proc1 because proc1 tries to write to closed proc2's stdin PIPE.

From my understanding, SIGPIPE would happen and proc1 would exit, regardless of closing proc1's stdout.

What do I miss?

---

Edit

After reading the post from @unutbu's comment......

I think the copied file descriptor(proc1.stdout) is writer PIPE, not reader PIPE. thus, there are two writer PIPE and one reader PIPE connected one another.

Therefore, SIGPIPE will be generated when proc2 exit because proc2 is only one process which has reader PIPE(will be closed when proc2 exit).

However, the above post seems to say that there are two reader PIPEs by copying proc1.stdout so SIGPIPE won't be generated even after proc2 exit because there still is another reader PIPE open. the below is the part of post.

So by closing p1.stdout immediately, you ensure that the only remaining filehandle reading from dmesg stdout is the grep process, and if that process were to exit, dmesg receives a SIGPIPE.

I am not saying that the post is wrong but I just want to fix my understanding. Thank you in advance.

Southerland

Does this post answer your question? – Bodleian

another link for reference #6047279 – Dayle

@unutbu, Thank you again, I edit my question after reading the post in your answer. – Southerland

Answer 1

proc1 = subprocess.Popen(['ps', 'cax'], stdout=subprocess.PIPE)

creates this a pipe between the parent process and proc1:

|        |         |       |
| parent |-<-----<-| proc1 |                   
|        | ^       |       |
           |                     
       p1.stdout   

p1.stdout is what the parent would read to obtain (stdout) output from proc1.

proc2 = subprocess.Popen(['grep', 'python'], stdin=proc1.stdout,
                         stdout=subprocess.PIPE, stderr=subprocess.PIPE)

connects a copy of the pipe from proc1 to proc2:

|        |         |       |         |       |
| parent |-<-----<-| proc1 |->----->-| proc2 | 
|        |         |       |         |       |

By calling p1.stdout.close(), we close the parent processes's side of the pipe:

|        |         |       |         |       |
| parent |       <-| proc1 |->----->-| proc2 | 
|        |         |       |         |       |

Now when proc2 terminates, its side of the pipe is also closed:

|        |         |       |         |       |
| parent |       <-| proc1 |->       | proc2 | 
|        |         |       |         |       |

The next time proc1 tries to write to the pipe, a SIGPIPE signal is generated, which allows proc1 to terminate since it knows no one is listening on the other end of its pipes.

Bodleian

Thank you very much!! I get clear. I had seemed to misunderstand on subprocess.PIPE. – Southerland

It may sound very basic but what is parent process in first command? – Dayle

@pankajmishra: The parent process is the process which is running the python script -- the one that calls subprocess.Popen. – Bodleian

Ok.Thanks for the answer – Dayle