The case to use PCOV to generate test coverage results
This article has been published a while ago.
If this is a technical article some information might be out of date. If something is terribly broken, let me know and I will update the article accordingly.
Test coverage is a metric which is often overvalued. Developers often assume they have to get to 100% coverage in their applications at all costs.
I think this is the wrong mindset. Not every single line of your application has to be tested. Writing Unit tests to test that a method getEmail() actually returns an email address is in my opinion, wasted time. (For packages, I agree that a high test coverage can be benefitial. It's usually also easier to do.)
However, knowing how much of your application is covered by tests can be a good indicator in which shape your project is. At work™, we recently started generating code coverage for all our projects.
In a particular large project (+1500 tests) the addition of running the test suite with xdebug enabled slowed the CI run down quite a bit. Normally the test suite takes 2 to 3 minutes to run. With xdebug enabled it took ~30 minutes. 😱
We're using GitHub Actions to run the test suite. In particular, we're using the shivammathur/setup-php-Action to create the correct PHP environment.
While figuring out, how we could speed up the CI run, I've stumbled upon the PCOV-option of the Action. PCOV is a driver built for generating code coverage. One of its main claims is, that it's much faster than xdebug. (Note: There will be a differerence in the coverage reporting between PCOV and xdebug (see here)).
Is it faster? Let's try it out.
These are the changes I had to make to our workflow. Instead of using coverage: xdebug we're now using coverage: pcov.
In addition, I had to set ini-values: pcov.directory=app. This tells the driver where our application code lives.
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
  php-version: 7.4
  extension: dom, curl, libxml, mbstring, zip, pcntl, pdo, sqlite, pdo_sqlite, bcmath, soap, intl, gd, exif, iconv, imagick
-  coverage: xdebug
+  coverage: pcov
+  ini-values: pcov.directory=app
And this is the entire GitHub Actions workflow we're using right now.
name: Tests (with Coverage)
on: push
jobs:
    tests:
        runs-on: ubuntu-latest
        steps:
            - name: Checkout Code
              uses: actions/checkout@v2
            - name: Setup PHP
              uses: shivammathur/setup-php@v2
              with:
                  php-version: 7.4
                  extension: dom, curl, libxml, mbstring, zip, pcntl, pdo, sqlite, pdo_sqlite, bcmath, soap, intl, gd, exif, iconv, imagick
                  ini-values: pcov.directory=app
                  coverage: pcov
            - name: Install Composer Dependencies
              run: composer install --prefer-dist --no-interaction --no-suggest
            - name: Copy Environment File
              run: cp .env.example .env
            - name: Generate Application Key
              run: php artisan key:generate
            - name: Execute PHPUnit Tests
              run: vendor/bin/phpunit --coverage-text --coverage-clover=coverage.xml
We've pushed the change in the mentioned work project and the time for running the test suite dropped significantly. From ~30 minutes to the ususal 2 to 3 minutes!
I was so happy to report this to the rest of the team. And I think it's also amazing, how low effort the change was. Actions like setup-php makes working with and tweaking Continous Integration workflows on GitHub so much easier.