Nesting 'WITH' statements in Python

Question

It turns out that "with" is a funny word to search for on the internet.

Does anyone knows what the deal is with nesting with statements in python?
I've been tracking down a very slippery bug in a script I've been writing and I suspect that it's because I'm doing this:

with open(file1) as fsock1:
    with open(file2, 'a') as fsock2:
        fstring1 = fsock1.read()
        fstring2 = fsock2.read()

Python throws up when I try to read() from fsock2. Upon inspection in the debugger, this is because it thinks the file is empty. This wouldn't be worrisome except for the fact that running the exact same code in the debugging interperter not in a with statement shows me that the file is, in fact, quite full of text...

I'm going to proceed on the assumption that for now nesting with statements is a no-no, but if anyone who knows more has a different opinion, I'd love to hear it.

Pare

Answer 1

I found the solution in python's doc. You may want to have a look at this (Python 3) or this (Python 2)

If you are running python 2.7+ you can use it like this:

with open(file1) as fsock1, open(file2, 'a') as fsock2:
    fstring1 = fsock1.read()
    fstring2 = fsock2.read()

This way you avoid unnecessary indentation.

Mazurka

Is there any way to run with f() as x, g(x) as y: ...? – Currency

Answer 2

AFAIK you can't read a file open with append mode 'a'.

Cinda

I was just about to comment on my own question upon realizing the same thing. I feel... silly now. – Pare

Answer 3

Upon inspection in the debugger, this is because it thinks the file is empty.

I think that happens because it can't actually read anything. Even if it could, when you append to a file, the seek pointer is moved to the end of the file in preparation for writing to occur.

These with statements work just fine for me:

with open(file1) as f:
    with open(file2, 'r') as g:   # Read, not append.
        fstring1 = f.read()
        fstring2 = g.read()

Note that use of contextlib.nested, as another poster suggested, is potentially fraught with peril here. Let's say you do this:

with contextlib.nested(open(file1, "wt"), open(file2)) as (f_out, f_in):
   ...

The context managers here get created one at a time. That means that if the opening of file2 fails (say, because it doesn't exist), then you won't be able to properly finalize file1 and you'll have to leave it up to the garbage collector. That's potentially a Very Bad Thing.

Grazier

"... Then you won't be able to properly finalize ..." This is untrue. Based on the Python docs, the second with statement will not execute until the first one has already begun. The compound with statement is semantically equivalent to nested with statements. – Fraternity

Answer 4

As of python 3.10 you can do it like this

with (
    Something() as example1,
    SomethingElse() as example2,
    YetSomethingMore() as example3,
):
...

this can be helpful in pytests when you want to do nested patches in some autouse fixture like so

from unittest.mock import patch

import pytest

@pytest.fixture(scope="session", autouse=True)
def setup():
    with (
        patch("something.Slow", MagicMock()) as slow_mock,
        patch("something.Expensive") as expensive_mock,
        patch("other.ThirdParty", as third_party_mock,
    ):
        yield

Ania

Answer 5

There is no problem with nesting with statements -- rather, you're opening file2 for append, so you can't read from it.

If you do dislike nesting with statements, for whatever reason, you can often avoid that with the contextlib.nested function. However, it won't make broken code (e.g., code that opens a file for append and then tries to read it instead) work, nor will lexically nesting with statements break code that's otherwise good.

Ruddock

Answer 6

As for searching for "with", prefixing a word with '+' will prevent google from ignoring it.

Thevenot

This used to be true. Now you have to surround it with quotes, i.e. "with". – Crossgrained

This isn't an answer to the question at hand (yes I know this question is 10 years old) – Netsuke