Phpunit very slow when --coverage-html

Question

Im using Phpunit. If I just run my tests with:

phpunit --log-junit output.xml

this runs within a second. But if I want a code coverage:

phpunit --coverage-html ./report --log-junit output.xml

then its very slow, the phpunit sends "Configuration read from *.xml" and it hangs for a minute, then it start executing the tests

Boulevard

It will be slower building the code coverage data, what's the problem with that? You only build the code coverage report when you want it – Abigail

I understand that debugging makes it slowlier. But it is slowier 10000000x times... – Boulevard

I think this may be more an issue with the slowness of xDebug than with PHPUnit. The xdebug_start_code_coverage() function is called with the XDEBUG_CC_DEAD_CODE and XDEBUG_CC_UNUSED options, and "Enabling those options make code coverage drastically slower." – Rinker

@Rinker hit the nail on the head. I am currently working on a patch which makes this 10 times faster. You can track the progress here. github.com/sebastianbergmann/php-code-coverage/pull/387 this will be merged into version 3.0 of php code coverage (supporting php >=5.6) . I may also fork this version and make it compatible with php5.5 – Dellinger

Answer 1

By default, PHPUnit will evaluate the coverage of all files in your configured whitelist, even when you run PHPUnit for a single test.

If you have a lot files in your whitelist, this can add a LOT of time to the generation of the code coverage.

You can speed things up by configuring PHPUnit to generate code coverage only for the files you have written/execute tests for, by setting the addUncoveredFilesFromWhitelist attribute to false.

<phpunit>
    <!-- ... -->
    <filter>
        <whitelist addUncoveredFilesFromWhitelist="false">
            <!-- ... -->
        </whitelist>
    </filter>
</phpunit>

With this setting disabled, you should see that the resulting code coverage files only describe the files you ran the tests for.

Note that the PHPUnit documentation suggests addUncoveredFilesFromWhitelist is false by default, but on version 5.5 it appears to be true by default.

Holbrook

Answer 2

Just to increment the last answer. Usually, it happens because PHPunit see all the files and you need to tell them to not do it, like this follow example:

Take a look to my filter node into phpunit.xml like this

<filter>
    <whitelist>
        <directory>../src</directory>
    </whitelist>
</filter>

I need to add addUncoveredFilesFromWhitelist equal to false and add all of the files which I need to exclude, like this:

<filter>
   <whitelist addUncoveredFilesFromWhitelist="false">
        <directory suffix=".php">../src</directory>
        <exclude>
            <directory>../vendor</directory>
            <directory>../anotherpath</directory>
            <directory>../src/Modules/*/Fixture</directory>
            <file>../src/Modules/*/Bootstrap.php</file>
        </exclude>
    </whitelist>
</filter>

Pay attention to my exclude list where I exclude the "Fixture" directory and Bootstrap.php file for all modules

After did this simple changes my unit tests change from 10 minutes to 3 minutes, so enjoy :)

Revile

Answer 3

When running phpunit with code coverage, the thing that impacts the speed the most is the fact that for each test method, phpunit uses xdebug with XDEBUG_CC_UNUSED and XDEBUG_CC_DEAD_CODE, which checks for dead and unused code in every file that is touched during that particular test execution. This includes vendor classes, test classes, phpunit framework classes and any other classes outside of your whitelist. phpunit only actually cares about the subjects under test (or classes in your whitelist).

I am in the process of tweaking code coverage to be smarter about running xdebug with the 2 mentioned flags on, so it only bothers checking out the dead and unused code of the files that phpunit cares about. You can check the progress here. https://github.com/sebastianbergmann/php-code-coverage/pull/387 and https://github.com/sebastianbergmann/php-code-coverage/pull/433 (merged on May 4, 2016 for PHPUnit 5.x.)

Dellinger

Answer 4

This is normal behaviour.

Think about what PHPUnit is doing:

It's running your tests, tracking every execution of every line of code, then taking all that raw data (the number of times each line was executed) and building a report by reading your code and reformatting it as HTML, supplemented with all that execution data.

It's not surprising that it takes a long time.

Driveway

but it didnt even start. It hangs on after starting and before sending dots, thats what I found strange – Boulevard

Not sure what you mean. In your original question you said it pauses for a minute, then starts running the tests. Assuming you literally mean "one minute", that doesn't surprise me. When I'm running tests, I find the same thing - not a full minute, but maybe you have a slower computer than I do, or more lines of code, etc. Although I know nothing about the internals of PHPunit, it wouldn't surprise me to learn that it starts by indexing your code. – Driveway

Does it have to index the code each time though? It seems to me that there should be some kind of smart caching in place--but maybe in this case that just isn't possible or wouldn't be helpful. – Rinker

Answer 5

phpdbg is more than time faster than xdebug

For more info please look on below link https://hackernoon.com/generating-code-coverage-with-phpunite-and-phpdbg-4d20347ffb45

Installation processes for linux:

sudo apt-get update -y
sudo apt-get install -y php-phpdbg

For more info please look on below link https://zoomadmin.com/HowToInstall/UbuntuPackage/php-phpdbg

To generate code coverage

phpdbg -qrr vendor/bin/phpunit --coverage-html ./var/log

I hope it will be very usefull to every one.

Beckman

Answer 6

In my case, my phpunit.xml was :

<phpunit>
    ...
    <coverage
            cacheDirectory=".phpunit.cache/code-coverage"
            includeUncoveredFiles="true">
    </coverage>
    ...
</phpunit>

I just needed to turn it off :

<phpunit>
    ...
    <coverage
            cacheDirectory=".phpunit.cache/code-coverage"
            includeUncoveredFiles="false">
    </coverage>
    ...
</phpunit>

Unsuspecting