The case to use PCOV to generate test coverage results
This article has been published a while ago.
If this is a technical article some information might be out of date. If something is terribly broken, let me know and I will update the article accordingly.
Test coverage is a metric which is often overvalued. Developers often assume they have to get to 100% coverage in their applications at all costs.
I think this is the wrong mindset. Not every single line of your application has to be tested. Writing Unit tests to test that a method getEmail() actually returns an email address is in my opinion, wasted time. (For packages, I agree that a high test coverage can be benefitial. It's usually also easier to do.)
However, knowing how much of your application is covered by tests can be a good indicator in which shape your project is. At work™, we recently started generating code coverage for all our projects.
In a particular large project (+1500 tests) the addition of running the test suite with xdebug enabled slowed the CI run down quite a bit. Normally the test suite takes 2 to 3 minutes to run. With xdebug enabled it took ~30 minutes. 😱
We're using GitHub Actions to run the test suite. In particular, we're using the shivammathur/setup-php-Action to create the correct PHP environment.
While figuring out, how we could speed up the CI run, I've stumbled upon the PCOV-option of the Action. PCOV is a driver built for generating code coverage. One of its main claims is, that it's much faster than xdebug. (Note: There will be a differerence in the coverage reporting between PCOV and xdebug (see here)).
Is it faster? Let's try it out.
These are the changes I had to make to our workflow. Instead of using coverage: xdebug we're now using coverage: pcov.
In addition, I had to set ini-values: pcov.directory=app. This tells the driver where our application code lives.
name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: 7.4
extension: dom, curl, libxml, mbstring, zip, pcntl, pdo, sqlite, pdo_sqlite, bcmath, soap, intl, gd, exif, iconv, imagick
coverage: xdebug
coverage: pcov
ini-values: pcov.directory=app
And this is the entire GitHub Actions workflow we're using right now.
name: Tests (with Coverage)
on: push
jobs:
tests:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v2
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: 7.4
extension: dom, curl, libxml, mbstring, zip, pcntl, pdo, sqlite, pdo_sqlite, bcmath, soap, intl, gd, exif, iconv, imagick
ini-values: pcov.directory=app
coverage: pcov
- name: Install Composer Dependencies
run: composer install --prefer-dist --no-interaction --no-suggest
- name: Copy Environment File
run: cp .env.example .env
- name: Generate Application Key
run: php artisan key:generate
- name: Execute PHPUnit Tests
run: vendor/bin/phpunit --coverage-text --coverage-clover=coverage.xml
We've pushed the change in the mentioned work project and the time for running the test suite dropped significantly. From ~30 minutes to the ususal 2 to 3 minutes!
I was so happy to report this to the rest of the team. And I think it's also amazing, how low effort the change was. Actions like setup-php makes working with and tweaking Continous Integration workflows on GitHub so much easier.
