Create a temporary FIFO (named pipe) in Python?

Question

How can you create a temporary FIFO (named pipe) in Python? This should work:

import tempfile

temp_file_name = mktemp()
os.mkfifo(temp_file_name)
open(temp_file_name, os.O_WRONLY)
# ... some process, somewhere, will read it ...

However, I'm hesitant because of the big warning in Python Docs 11.6 and potential removal because it's deprecated.

EDIT: It's noteworthy that I've tried tempfile.NamedTemporaryFile (and by extension tempfile.mkstemp), but os.mkfifo throws:

OSError -17: File already exists

when you run it on the files that mkstemp/NamedTemporaryFile have created.

Melodic

As you mention, since os.mkfifo throws an error if the file exists, there is no security hole using mkstemp or even a totally deterministic filename (besides a bad feeling and a risk of someone naively copying your code). – Coppins

Question: How do you plan to communicate the name of the pipe to the other processes? How is this channel secured? – Cinch

@Joe: You're right. I hadn't thought of that. @Steven Huwig: Both processes are spawned by the same parent, though I might at some point move to a new fifo for that, too. Why do you ask? – Melodic

Answer 1

os.mkfifo() will fail with exception OSError: [Errno 17] File exists if the file already exists, so there is no security issue here. The security issue with using tempfile.mktemp() is the race condition where it is possible for an attacker to create a file with the same name before you open it yourself, but since os.mkfifo() fails if the file already exists this is not a problem.

However, since mktemp() is deprecated you shouldn't use it. You can use tempfile.mkdtemp() instead:

import os, tempfile

tmpdir = tempfile.mkdtemp()
filename = os.path.join(tmpdir, 'myfifo')
print filename
try:
    os.mkfifo(filename)
except OSError, e:
    print "Failed to create FIFO: %s" % e
else:
    fifo = open(filename, 'w')
    # write stuff to fifo
    print >> fifo, "hello"
    fifo.close()
    os.remove(filename)
    os.rmdir(tmpdir)

EDIT: I should make it clear that, just because the mktemp() vulnerability is averted by this, there are still the other usual security issues that need to be considered; e.g. an attacker could create the fifo (if they had suitable permissions) before your program did which could cause your program to crash if errors/exceptions are not properly handled.

Cannoneer

Should the os.rmdir(tmpdir) not be outside the the try-else block? – Remove

You mean except OSError as e. – Bostick

@TimDierks: It is true that the new syntax is more readable and the only one supported in Python 3, but the old one is valid for all Python 2.x versions, and the new one needs at least Python 2.6. – Bookman

Answer 2

You may find it handy to use the following context manager, which creates and removes the temporary file for you:

import os
import tempfile
from contextlib import contextmanager


@contextmanager
def temp_fifo():
    """Context Manager for creating named pipes with temporary names."""
    tmpdir = tempfile.mkdtemp()
    filename = os.path.join(tmpdir, 'fifo')  # Temporary filename
    os.mkfifo(filename)  # Create FIFO
    try:
        yield filename
    finally:
        os.unlink(filename)  # Remove file
        os.rmdir(tmpdir)  # Remove directory

You can use it, for example, like this:

with temp_fifo() as fifo_file:
    # Pass the fifo_file filename e.g. to some other process to read from.
    # Write something to the pipe 
    with open(fifo_file, 'w') as f:
        f.write("Hello\n")

Squamosal

Answer 3

How about using

d = mkdtemp()
t = os.path.join(d, 'fifo')

Rollins

That suffers the same security issues as mktemp. Thanks though, Brendan. – Melodic

It only has the same security issue if the user has already compromised the account running the script (the directory is 0700). In which case, they can probably do much worse. – Coppins

Oop - I was wrong. This doesn't have the same security issue as mktemp because mkfifo() balks if the file exists (i.e. there's no chance for a MiM attack). This would work. – Melodic

Answer 4

If it's for use within your program, and not with any externals, have a look at the Queue module. As an added benefit, python queues are thread-safe.

Merely

Thanks Nilamo. Unfortunately it's not for internal communication. I'm using Queues elsewhere, though. Thanks for the suggestion. – Melodic

Not a problem, just throwing the 'use the simplest possible thing that could work' flag. If you actually need the fifo, then this answer doesn't help at all. – Merely

Answer 5

Effectively, all that mkstemp does is run mktemp in a loop and keeps attempting to exclusively create until it succeeds (see stdlib source code here). You can do the same with os.mkfifo:

import os, errno, tempfile

def mkftemp(*args, **kwargs):
    for attempt in xrange(1024):
        tpath = tempfile.mktemp(*args, **kwargs)

        try:
            os.mkfifo(tpath, 0600)
        except OSError as e:
            if e.errno == errno.EEXIST:
                # lets try again
                continue
            else:
                raise
        else:
           # NOTE: we only return the path because opening with
           # os.open here would block indefinitely since there 
           # isn't anyone on the other end of the fifo.
           return tpath
    else:
        raise IOError(errno.EEXIST, "No usable temporary file name found")

Myeloid

Answer 6

Why not just use mkstemp()?

For example:

import tempfile
import os

handle, filename = tempfile.mkstemp()
os.mkfifo(filename)
writer = open(filename, os.O_WRONLY)
reader = open(filename, os.O_RDONLY)
os.close(handle)

Delimitate

os.mkfifo throws an OSError 17: File Exists. Thanks though. – Melodic