Redirection in Django command called from another command results in extraneous newlines. How can I fix this?

Question

The Setup

For the purpose of illustrating the problem, I have created these commands in my project:

foo.py:

from django.core.management.base import BaseCommand
from django.core.management import call_command

class Command(BaseCommand):

    def handle(self, *args, **options):
        self.stdout.write("foo")
        # I pass `self.stdout` here explicitly because if `foo` is
        # redirected, I want `baz` redirected too.
        call_command('baz', stdout=self.stdout)

baz.py:

from django.core.management.base import BaseCommand
from django.core.management import call_command

class Command(BaseCommand):

    def handle(self, *args, **options):
        # This could be reduced to one call to self.stdout.write
        # but this code is meant to minimally reproduce what happens in a 
        # complex command where multiple self.stdout.write calls are
        # made. If the code here were replaced with a single call, it 
        # would cease to reproduce the issue.
        self.stdout.write("baz ", ending='')

        # Imagine a lot of stuff happening here with conditionals and
        # loops.
        self.stdout.write("baz")

Actual Behavior

I run foo like this:

./manage.py foo

And I get this output to the console:

foo
baz 
baz

Desired Behavior

What I want is the output to the console to be:

foo
baz baz

Note that when I invoke baz directly with ./manage.py baz, I get this output:

baz baz

There is no newline between the two "baz". I want the same layout when baz is invoked through foo.

Diocese

Linking back: Spurious newlines added in Django management commands – Straightforward

Answer 1

The Problem

The reason it is not working is that Django uses an OutputWrapper object to wrap whatever is passed as the stdout argument of a Command. This object becomes self.stdout in the command's methods. (This is true in Django 1.8 and as far as I can tell as far back as Django 1.5.)

OutputWrapper has a write method which by default adds a newline to what is written to the output. You can turn it off with ending='', which is what you do and works fine when baz is invoked directly. However, the OutputWrapper object does not expect that it will be wrapping another OutputWrapper object. When your baz code is called through foo and executes self.stdout.write("baz ", ending='') it calls write on the object it is wrapping, but it does not forward the ending='' parameter. So the OutputWrapper that was created for foo is called without ending='' and a newline is output to the console.

Solutions

The solution I prefer is to replicate in my code exactly what Django does when it decides what OutputWrapper should wrap:

class Command(BaseCommand):

    def handle(self, *args, **options):
        self.stdout.write("foo\n")
        call_command('baz', stdout=options.get('stdout', sys.stdout))

The stdout=options.get('stdout', sys.stdout) bit will pass sys.stdout if no stdout keyword parameter was given to foo. Otherwise, it forwards the stdout keyword parameter. You can do the same with stderr by changing all instances of stdout to stderr.

Another way would be to set the ending of the OutputWrapper to '', like this:

class Command(BaseCommand):

    def handle(self, *args, **options):
        self.stdout.ending = ''
        self.stdout.write("foo\n")
        call_command('baz')

You then have to write your command while keeping in mind that you always have to output newlines explicitly: this is why we now have self.stdout.write("foo\n"), with a newline at the end of the string. This has the advantage that anything that baz outputs appears immediately on the console, so if it hangs after some output, you at least have something to work with. However, OutputWrapper is not a class that has been documented for direct use by Django projects. This solution basically uses an API that could change without warning in newer releases of Django.

Diocese