testing multiple folders

Question

I use PHPUnit 3.5.12, netbean 6.9, and git submodules in my project.

So my folder architecture looks like that:

lib/
lib/submodule1
lib/submodule1/src
lib/submodule1/tests
lib/submodule2
lib/submodule2/src
lib/submodule2/tests
src/
tests/

Considering that my main test folder (with phpunit_netbean.xml and bootstrap.php) is in the /tests/ folder; How can I be able to run the tests in /lib/*/tests/ too ?

I've look at testsuite, but I'm unable to get it working. So far I've tried the following configuration in my tests/phpunit_netbean.xml file:

<?xml version="1.0"?>
<phpunit
    bootstrap="./bootstrap.php"
    strict="true"
    stopOnError="false"
    stopOnFailure="false"
    stopOnIncomplete="false"
    stopOnSkipped="false"
    colors="false"
    verbose="true"
    >

    <testsuites>
        <testsuite name="modules">
            <directory>../lib/*</directory>
        </testsuite>
    </testsuites>

</phpunit>

And when I hit ALT+F6 in Netbean, I only have the tests from /tests that are run. Same thing with:

/tests$ phpunit -c phpunit_netbean.xml --testdox ./
enter code here

Also, I've tried this:

/tests$ phpunit -c phpunit_netbean.xml --testdox --loader modules ./
PHPUnit 3.5.12 by Sebastian Bergmann.

Could not use "modules" as loader.

Altair

You edited while i answered but i think it might still be avalid point. What do you need the --loader for? (It kinda should just work) – Hierodule

Well I was thinking that the --loader parameter could tell phpunit what testsuite to use, but no. – Altair

Answer 1

Have a look at the Appendix for your PHPUnit config file. You can tell PHPUnit which folders to include:

<testsuites>
    <testsuite name="Submodules">
        <directory suffix="Test.php">../lib/*</directory>
    </testsuite>
</testsuites>

When you run this, PHPUnit should recursively iterate over all folders in lib and consider all files ending in Test.php as UnitTests. The name of the testsuite is irrelevant. To ease authoring of the XML file, consider using my phpunit-schema file.

There is some additional information to this in the PHPUnit Manual Chapter: Composing a Test Suite Using the Filesystem . Netbeans has an input dialog in which you can specify the path to the phpUnit.xml and any custom TestSuite Class.

Here is an example project: https://github.com/gooh/sample

C:\Users\Gordon\Desktop\demo\tests>phpunit --testdox
PHPUnit 3.5.13 by Sebastian Bergmann.

A
 [x] One

B
 [x] One

My
 [x] One

Cringle

Hi Gordon, Yup, I've read that chapter (before posting), but I can't get it to work. I'll edit my question to add some info of what I've tried. – Altair

I added the suffix="Test.php", and even with a working lib/News/tests/NewsTest.php file, it doesn't get scanned at all ( by netbean and phpunit in shell ) No errors reported. – Altair

@FMaz it works on my machine. I'll add a sample project at github – Cringle

Ok, obviously I've something wrong because you've made exactly what I want to do... hum... let me take a closer look. – Altair

Haha, is it possible that the testssuites are only executed if no <directory> is passed to phpunit ? If so I'll need to find out how to configure netbean not to pass a path.. – Altair

Answer 2

Let's create a testcase:

mkdir project
mkdir project/tests
mkdir project/modules/
mkdir project/modules/a/
mkdir project/modules/a/tests/
mkdir project/modules/b/
mkdir project/modules/b/tests/

echo "<?php class MyTest extends PHPUnit_Framework_TestCase { public function testOne() { \$this->assertTrue(true); } } " > project/tests/MyTest.php
echo "<?php class MyTwoTest extends PHPUnit_Framework_TestCase { public function testOne() { \$this->assertTrue(true); } } " > project/modules/a/tests/MyTwoTest.php
echo "<?php class MyThreeTest extends PHPUnit_Framework_TestCase { public function testOne() { \$this->assertTrue(true); } } " > project/modules/b/tests/MyThreeTest.php


tree
.
`-- project
    |-- modules
    |   |-- a
    |   |   `-- tests
    |   |       `-- MyTwoTest.php
    |   `-- b
    |       `-- tests
    |           `-- MyThreeTest.php
    `-- tests
        `-- MyTest.php

---

If you now just run phpunit project it will execute ALL the tests in those folders. So no need for any further configuration.

The only thing that could bite you if you have lasses in your /src/ folders that end in *Test.php, if you don't do that you will not have any issues as far as i can tell.

phpunit project/
PHPUnit 3.5.11 by Sebastian Bergmann.

...

Time: 0 seconds, Memory: 2.75Mb

Hierodule

The submodule tests are contained in their own folder. In the example you given, all your test belong in the /tests/ folder. That part is not a problem. It become a problem when I want more than a simple recurtion. I want to be able to define more than 1 tests folder to scan. Let say you add a folder there: mkdir -p lib/submodule1/tests ... how to you tell phpunit to scan that folder too when doing a full project tests run ? – Altair

My not just take the folder above your main test folder as the root and let phpunit scan everything recursively? (It should work, but you are right, my example is kinda missing your point. I'll edit) – Hierodule

Ok. Does this look more like it? I've tried to create a working phpunit.xml with a glob pattern but that didn't work out so i took the "easier" why of you letting phpunit do the work. If you want to have each subfolder as a <directory> in your phpunit.xml i don't know away around of creating a script that creates the phpunit.xml for you :( – Hierodule

That's a brilliant idea, and it solve a part of the problem. However, putting this as the default behavior makes the tests slow as I must alway scan the whole project. And netbean can't figure out what's the corresponding test file for a specific class of a submodule. Example: if I edit a submodule file, I want to be able to run the test of that submodule file, not the whole project. – Altair

Heh, yeah. For developing the modules each would need to be its own project since netbeans only is aware of "one test folder". At least we didn't find a way around that issue with netbeans – Hierodule

The problem I have with that is when a submodule is like a plugin. It can't be run alone (without an context (an application) )... As least I have less the impression of asking a dumb question ;) – Altair

If build a bootstrap that creates something that looks like the application for testing purposes since we don't want to bootstrap the whole app just to test the modules. But ok. It seemed to have worked out for you, hope my answer also helped ;) – Hierodule