PHPSpec and coverage report

Question

Does anyone know a method to generate coverage reports from PHPSpec tests?

I thought about xdebug, but as far as I know it cant generate reports for jenkins.

Industrious

Answer 1

As it stands (1.4.0) it doesn't yet support code coverage. Happy to hear your feedback on this. Below is my opinion on code coverage.

PHPSpec is a BDD framework. If you were doing BDD you would describe the behaviour of your class prior to writing your class. If you did that way the relevant behaviour of your classes would all be properly covered with "tests".

Code coverage tools and metrics are useful for legacy code (code you wrote without specs/tests). You can use a tool like that to try and get up to a point where you can continue TDD and have the benefit of being protected from regression.

In general that approach isn't really as effective as describing the behaviour first (TDD). A single method may be simple enough to respond to more than one required behaviour. You know that when you are TDDing, you keep refactoring in the process, deleting unneeded code. You end up with 10 specs (tests) hitting the same lines of code, all describing different needed behaviour, all useful to understand the code.

One of the problems with the word "test" is that it makes people think TDD is about verification. It's not. It's about communication. StoryBDD is communication between stakeholders and SpecBDD is communication between classes. Simple, living, just enough documentation.

Code coverage done to guarantee you have tested your code is a fallacy, a poor metric at best. Unfortunately people think testing structure is more important than testing behaviours. That's why BDD was born, to help bring the focus back on the right track. Making sure this part of the code is tested is fake because that part of the code can do more than one thing, it should, if it's nicely refactored. Also you will end up testing stuff like accessors and modifiers and constructors, etc.

But I am open to hear the community on this. I can see where Code Coverage could be useful. Plus since Sebastian Bergmann nicely modularised it out of PHPUnit I could reuse it in PHPSpec. I would prefer you wrote your specs first. You get 100% code coverage of your relevant behaviour for free. In my views, that is what matters for the most part.

Emia

Your aproach as more exprerienced person is very valuable for me. I'm starting to incorporate BDD in my team, that's why I so much care about doing it right and gather usefull stats :) – Industrious

It's great approach for new projects, but what about if we want to implement BDD in existing legacy one? Have You any experience in this? Is there any sense to write Spec for existing code? In my opinion for old code there should be unit tests for new one specs. – Creamy

Great answer Marcello. Still, code coverage gives reassurance, faith where it lacks :) – Kimball

it could also be useful when removing features – Turtledove

When you write code at first every line of code is written to satisfy specification. So there is no dead code. But later on business requirements might change, and when they do the specification also has to change. So then the production code changes - and some code might end up being dead. Code coverage would help hunt it down. Also like they said, it would be helpful to write specification for existing code. – Demello

CI is the use case here, you surely want that a whole team writes tests and not just the few developers that see the point – Fairman

Answer 2

Use the Code Coverage extension to PHPSpec to generate clover.

If you would like to merge coverage data from PHPSpec and other tools (eg) PHPUnit, then use the PHP_CodeCoverage output format with the phpcov tool in merge mode.

Examples:

# phpspec.yml
extensions:
    - PhpSpec\Extension\CodeCoverageExtension

code_coverage:
    output: /tmp/coverage/phpspec.phpcoverage
    format: php

# phpunit.xml
<logging>
   <log type="coverage-php" target="/tmp/coverage/phpunit.phpcoverage" />
</logging>

# from the command line
phpcov merge --clover coverage.xml /tmp/coverage

That will give you coverage from both tools in a final clover format, suitable for like Jenkins.

Hatten

Answer 3

I think a code-coverage generator would be usefull for legacy systems that is in the process of being subjected to BDD style testing as you will be able to tell what code is not tested. I for one would appreciate such a feature in PHPSpec.

Drucie