Fast PHPUnit and XDebug code coverage

What is PHP Unit?

PHPUnit is a popular unit testing framework for the PHP programming language. Put simply, if you’re writing PHP, then you had better be testing your projects with PHPUnit.

Talk to one of our digital experts

Tom Houdmont

Head of Business Solutions

Do you have an idea or a project you need support with?

Tom leads Box UK’s Business Solutions team and has over 15 years experience in the web industry.  Tom is passionate about creating impactful solutions that solve real problems and deliver the outcomes our clients need.

Book a meeting with Tom

Or call us on 020 8098 2093

What is XDebug?

A debugger and profiler tool for PHP, you’d also better be coupling your unit testing with XDebug, to take full advantage of all the benefits pragmatic code coverage gives you.

So what’s the problem?

Unfortunately though, with XDebug comes a performance cost, and depending on the size of your codebase it may not be a small one. This is a short post about a useful tweak we made around this to our Continuous Integration (CI) setup at Box UK.

We use the Symfony 2 framework extensively with our PHP projects. We always strive for good code coverage levels, but having XDebug enabled on our Jenkins slaves was having dramatic effects on our builds times; in some cases driving it up from a few minutes to nearly half an hour! When you thrive on short feedback loops, this is just too long. You’ve probably moved onto your next task by the time the dreaded build fail notification comes along.

First attempt

Our first stab at this setup was simply to have two PHP Jenkins slaves. We’d then create one build to run our tests which would target the non-XDebug slave, and a second job to run code coverage which would target the slave with XDebug enabled. This worked fine, but felt like more moving parts than we needed… so let’s see if we can do a little better.

PHP_INI_SCAN_DIR

In many years with PHP I’d never come across this environment variable, but it allows specifying a single folder where PHP will look for .ini files. As many as you like. Brilliant!So we figured the simplest solution would be to build a CI slave with XDebug installed, but disabled by default. We then planned to have this enabled explicitly by jobs that required it.

Ant target

We usually use an Ant build file, created originally using Sebastian Bergmann’s excellent PHP Template for Jenkins Jobs. To this we added a code coverage target to enable XDebug when needed. Here’s the snippet…

<target name="phpunit-coverage" description="Runs PHPUnit with XDebug enabled">
    <copy todir="${dir.phpd}">
        <fileset dir="${dir.phpd.original}"/>
    </copy>
    <move file="${dir.phpd}/xdebug.ini.disabled"
          tofile="${dir.phpd}/xdebug.ini"/>
    <!-- xml formatting intentional, don’t want whitespace at start of line -->
    <echo file="${dir.phpd}/coverage.ini">
memory_limit = 1536M
</echo>
    <exec executable="phpunit" failonerror="true">
        <env key="PHP_INI_SCAN_DIR" path="${dir.phpd}"/>
    </exec>
</target>

As you can see, what happens is we copy all PHP system configuration from the default location $dir.phpd.original (/etc/php.d) to our temporary build configuration directory $dir.phpd ($WORKSPACE/build/php.d). We then rename the disabled xdebug.ini.disabled so it will get picked up when PHP runs.

This then means that our standard build will run without XDebug, nice and fast. And that when it comes to code coverage time XDebug will be enabled and it’ll all Just Work. Just what we wanted!

Other configuration

This technique obviously isn’t limited to just enabling XDebug. Another problem we encountered was that while generating code coverage a LOT of memory is required. We didn’t want to up the system configured limit just to appease this.

<echo file="${dir.phpd}/coverage.ini">
memory_limit = 1536M
</echo>

So as you can see from the above example, we also write another coverage.ini file which ups the memory limit – but of course only in the cases where we’re generating code coverage. You can of course customise anything you like here, and have it only apply for individual targets!

Conclusion

This simple tweak has simplified our CI setup, and given us more per-job configurability without affecting our default server configuration. It seems all good so far, and we hope you find it as useful as we have.

Pete Withers-Jones

Head of Development

Pete Withers-Jones leads Box UK’s Development Practice, and has over 20 years of experience in software development, working across various industries and technologies.