Behat eating memory

Question

I'm using Behat for testing a Symfony2 application. Whilst each Feature test runs happily when run in isolation, trying to run the whole test suite in one go leads to PHP running out of memory - even when the memory_limit is set to 2GB and higher.

By echoing the current memory usage at the end of each Feature, I can see that memory usage increases by between 20 and 50MB for each feature that runs.

So, my question is, "is there anything I can do to free up memory after each Feature has run?" It appears that each Feature is booting up another Symfony application, so, my ideal solution would be to destroy each Symfony application (assuming that is what is happening) after each Feature has run using an @AfterFeature hook.

Updated to add: We're using Symfony 2.3.7 and Behat 2.5.0.

Updated to add: A typical use case is;

• use Doctrine to put the system / entities into a known state;
• simulate a user clicking on various links, filling in form fields etc;
• use Doctrine to check that the entities are the expected state

Zarla

Which behat and symfony 2 version do you use ? Thanks – Fleer

What kind of things do you do in the context files? Do you use doctrine to populate the db, etc? I never had issues with Behat itself, it might be something specific you do that makes it eat memory. – Mashburn

Same problem, same versions, and behat/mink-browserkit-driver v1.1.0 Current workaround is to increase the php memory limit, though risk running out of VM soon! – Fascinating

try to add execution of gc_collect_cycles() in after feature hook, does it help? – Verb

Added gc_collect_cycles() in the after feature hook, but it didn't help - each Feature still added 20-50MB to the memory usage. – Zarla

@AndrewBattye were you able to figure out the memory issue? I am keep getting PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 130968 bytes) when I am trying to use $this->iClick('url') in behat. – Rummy

Hi Adam - no, we never resolved it. The best plan we came up with was to write a batch file that ran each Behat feature in turn, rather than running the whole suite in one go. This allowed all the tests to run, but it meant that we had a bit of post-processing work to do with all the results files. – Zarla

Answer 1

Typically, PHP software is not written in such a way to be able to release memory. Instead, the software relies on the fact that it will likely only run for a second or two before terminating, thus clearing the memory.

When you run tests like this, you are likely hitting memory leaks in the main application. Add additional memory checks around the functions being called by the code, and then around the functions those functions call, etc., until you find the culprit.

In my experience, the problem will typically be the reuse of an object variable in a loop:

function f() {
    foreach ($list as $item) {
        $x = new C($item);
        $x->doStuff();
    }
}

Normally when "f" exits, all the memory is cleaned up. But PHP is stupid, so it figures this out by looking at the local variables or something, because only the last $x is going to get cleaned up. The ones created before it in that loop will just leak until the script exits.

If this is - in fact - the problem, you can fix it by using unset on the variable before using it again.

$x = new C($item);
$x->doStuff();
unset($x);

Ambrogino

Just to be clear, the problem is almost certainly going to be found in both your code and in Symfony. These problems are an absolute plague in almost all PHP code in the wild. I would venture to guess Behat is the last place you'd find this problem, as this is something that type of software is forced to deal with. – Ambrogino

I know this comment is from 2013, but afaik modern PHP versions use refcounting for garbage collection, that is a value in memory stores how many references, e.g., variables, point to it, and when a variable goes out of scope or stops pointing to it, the refcount is decremented. In your example therefore there is no need to unset $x – Silly

In theory yes. But in my testing it seems that PHP "forgets" the variable if it's re-used, for example in a loop. PHP doesn't clean anything until the function ends, and by that time it seems to have lost the reference. Using unset forces it to cleanup immediately. – Ambrogino