How to save pytest's results/logs to a file?

Question

I am having trouble trying to save -all- of the results shown from pytest to a file (txt, log, doesn't matter). In the test example below, I would like to capture what is shown in console into a text/log file of some sort:

import pytest
import os

def test_func1():
    assert True


def test_func2():
    assert 0 == 1

if __name__ == '__main__':

    pytest.main(args=['-sv', os.path.abspath(__file__)])

Console output I'd like to save to a text file:

test-mbp:hi_world ua$ python test_out.py
================================================= test session starts =================================================
platform darwin -- Python 2.7.6 -- py-1.4.28 -- pytest-2.7.1 -- /usr/bin/python
rootdir: /Users/tester/PycharmProjects/hi_world, inifile: 
plugins: capturelog
collected 2 items 

test_out.py::test_func1 PASSED
test_out.py::test_func2 FAILED

====================================================== FAILURES =======================================================
_____________________________________________________ test_func2 ______________________________________________________

    def test_func2():
>       assert 0 == 1
E       assert 0 == 1

test_out.py:9: AssertionError
========================================= 1 failed, 1 passed in 0.01 seconds ==========================================
test-mbp:hi_world ua$ 

Doubleacting

no exact answers till now – Chimp

Answer 1

It appears that all of your test output is going stdout, so you simply need to “redirect” your python invocation's output there:

python test_out.py >myoutput.log

You can also “tee” the output to multiple places. E.g., you might want to log to the file yet also see the output on your console. The above example then becomes:

python test_out.py | tee myoutput.log

Disbranch

Seems like tee is included in Windows 10. – Foochow

This is the correct answers based on what the OP ask for, but another useful answer is pytest --junitxml=./output.xml. This outputs a junit xml file which can be opened in a junit viewer such as xunit-viewer – Footway

Is it possible to show some kind of progress bar while my pytest is building up the file? Currently when I am doing the same it is taking some time to complete the test and writing it into the file. So, some kind of progress counter would help to check whether the program is running or not. – Lariat

@senarijit1618, look at this config – Breathy

The down side of the tee approach is the exit status of pytest get's eaten, since tee always returns 0. – Addressograph

Answer 2

I derive this from pastebin as suggest by Bruno Oliveira :

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""
Pytest Plugin that save failure or test session information to a file pass as a command line argument to pytest.

It put in a file exactly what pytest return to the stdout.

To use it :
Put this file in the root of tests/ edit your conftest and insert in the top of the file :

    pytest_plugins = 'pytest_session_to_file'

Then you can launch your test with the new option --session_to_file= like this :

    py.test --session_to_file=FILENAME
Or :
    py.test -p pytest_session_to_file --session_to_file=FILENAME


Inspire by _pytest.pastebin
Ref: https://github.com/pytest-dev/pytest/blob/master/_pytest/pastebin.py

Version : 0.1
Date : 30 sept. 2015 11:25
Copyright (C) 2015 Richard Vézina <ml.richard.vezinar @ gmail.com>
Licence : Public Domain
"""

import pytest
import sys
import tempfile


def pytest_addoption(parser):
    group = parser.getgroup("terminal reporting")
    group._addoption('--session_to_file', action='store', metavar='path', default='pytest_session.txt',
                     help="Save to file the pytest session information")


@pytest.hookimpl(trylast=True)
def pytest_configure(config):
    tr = config.pluginmanager.getplugin('terminalreporter')
    # if no terminal reporter plugin is present, nothing we can do here;
    # this can happen when this function executes in a slave node
    # when using pytest-xdist, for example
    if tr is not None:
        config._pytestsessionfile = tempfile.TemporaryFile('w+')
        oldwrite = tr._tw.write

        def tee_write(s, **kwargs):
            oldwrite(s, **kwargs)
            config._pytestsessionfile.write(str(s))
        tr._tw.write = tee_write


def pytest_unconfigure(config):
    if hasattr(config, '_pytestsessionfile'):
        # get terminal contents and delete file
        config._pytestsessionfile.seek(0)
        sessionlog = config._pytestsessionfile.read()
        config._pytestsessionfile.close()
        del config._pytestsessionfile
        # undo our patching in the terminal reporter
        tr = config.pluginmanager.getplugin('terminalreporter')
        del tr._tw.__dict__['write']
        # write summary  
        create_new_file(config=config, contents=sessionlog)


def create_new_file(config, contents):
    """
    Creates a new file with pytest session contents.
    :contents: paste contents
    :returns: url to the pasted contents
    """
    # import _pytest.config
    # path = _pytest.config.option.session_to_file
    # path = 'pytest_session.txt'
    path = config.option.session_to_file
    with open(path, 'w') as f:
        f.writelines(contents)


def pytest_terminal_summary(terminalreporter):
    import _pytest.config
    tr = terminalreporter
    if 'failed' in tr.stats:
        for rep in terminalreporter.stats.get('failed'):
            try:
                msg = rep.longrepr.reprtraceback.reprentries[-1].reprfileloc
            except AttributeError:
                msg = tr._getfailureheadline(rep)
            tw = _pytest.config.create_terminal_writer(terminalreporter.config, stringio=True)
            rep.toterminal(tw)
            s = tw.stringio.getvalue()
            assert len(s)
            create_new_file(config=_pytest.config, contents=s)

Recompense

Actually, I was in the process of packaging this pytest plugin and finally found the list of third-party plugins (pytest.org/latest/plugins_index/index.html) and I am not sure that what I did is not the same of pytest-capturelog or pytest-catchlog, the output of the plugin is surely not the same though it may provide the same information. If someone that know these plugins can confirm... If so, I may better just add the ability to get the same output from these plugins then the one above... – Recompense

Here the pypi package : pypi.python.org/pypi/pytest-session_to_file/0.1.0 – Recompense

Finally a working pypi package here : pypi.python.org/pypi/pytest-session2file/0.1.5 – Recompense

Would this work also if you invoke pytest.main()? I am trying to save the output of the pytest tests and it is amazing that almost all the answers here does not mention once the pytest.main() form. Thanks! – Celebes

I don't see why not... But I didn't try since the thing that with the thing that I test I didn't succeed in using pytest.main()... So I created my own test luncher script to reduce the boiler plate... Note, I never had time to update the pypi package but there is fix for python 3 in the github repo... – Recompense

⁺¹ for the snippet! I should note two things though: α) the code as is throws module '_pytest.config' has no attribute 'option on line path = config.option.session_to_file (however it does create pytest_session.txt despte the error), and β) the packaged version of the code does not work at all, it fails with write() argument must be str, not bytes. – Valladolid

Have you give a look at the github repo, that the purpose of the plugin to write things to file, the code above might not be up to date with the code base of the plugin : github.com/BuhtigithuB/pytest_session2file – Recompense

On a newer pytest this gives got an unexpected keyword argument 'stringio' – Valladolid

Answer 3

The pastebin internal plugin does exactly that, but sends the output directly to bpaste.net. You can look at the plugin implementation to understand how to reuse it for your needs.

Moulton

Answer 4

Here is a fixture in order for you to be able to do this, I used the pytest Cache feature in order to leverage a fixture that can be passed around to multiple test files, including distributed tests(xdist), in order to be able to collect and print test results.

conftest.py:

from _pytest.cacheprovider import Cache
from collections import defaultdict

import _pytest.cacheprovider
import pytest

@pytest.hookimpl(tryfirst=True)
def pytest_configure(config):
    config.cache = Cache(config)
    config.cache.set('record_s', defaultdict(list))

@pytest.fixture(autouse=True)
def record(request):
    cache = request.config.cache
    record_s = cache.get('record_s', {})
    testname = request.node.name
    # Tried to avoid the initialization, but it throws errors.
    record_s[testname] = []
    yield record_s[testname]
    cache.set('record_s', record_s)

@pytest.hookimpl(trylast=True)
def pytest_unconfigure(config):
    print("====================================================================\n")
    print("\t\tTerminal Test Report Summary: \n")
    print("====================================================================\n")
    r_cache = config.cache.get('record_s',{})
    print str(r_cache)

Use:

def test_foo(record):
    record.append(('PASS', "reason", { "some": "other_stuff" }))

Output:

====================================================================

        Terminal Test Report Summary:

====================================================================

{u'test_foo': [[u'PASS',u'reason', { u'some': u'other_stuff' } ]]}

Tope

Answer 5

Expanding on @Micah's helpful solution, as answered in this similar question, you can add to the filename both username and timestamp via:

pytest . -vv | tee ./logs/$(whoami)_"$(date +%Y%m%d_%H%M%S)".log

where

• . runs all tests in test/
• -vv returns very verbose results
• whoami resolves to your username on linux/windows
• the date section produces YYYYMMDD_HHMMSS

Dulcle