hpc-codecov

The hpc-codecov package contains an executable and library codes for generating Codecov JSON coverage report from .tix and .mix files made with hpc. The generated report is ready to be uploaded to Codecov with other tools such as Codecov Bash uploader.

The hpc-codecov executable can search .tix and mix files under the directories made by the cabal-install and stack build tools. The executable also has options to explicitly specify the file paths and directories for .tix and mix files, to support generating reports with test data made by other build tools than cabal-install and stack.

Installing

From Package Repository

hpc-codecov is available from Hackage and Stackage. To install with cabal-install, run:

$ cabal install hpc-codecov

To install with stack, run:

$ stack install hpc-codecov

Pre-compiled binaries

For Windows, MacOS, and Linux (with glibc and libgmp), pre-compiled binary executables are available here.

QuickStart

To illustrate an example, initializing sample project named my-project with cabal-install:

$ cabal --version
cabal-install version 3.4.0.0
compiled using version 3.4.0.0 of the Cabal library
$ cabal init --simple --tests --test-dir=test -p my-project

Directory contents look like below:

.
├── app
│   └── Main.hs
├── CHANGELOG.md
├── my-project.cabal
├── src
│   └── MyLib.hs
└── tests
    └── MyLibTest.hs

Run tests with coverage option:

$ cabal test --enable-coverage

Write Codecov coverage report to codecov.json:

$ hpc-codecov cabal:my-project-test -X my-project -o codecov.json

Show coverage report contents:

$ cat codecov.json
{"coverage":{"test/MyLibTest.hs":{"4":1}}}

Using in GitHub workflow

See hpc-codecov-action to generate Codecov coverage report from GitHub workflow.

Examples

Showing help

Show usage information:

$ hpc-codecov --help

Project using cabal-install

Search under directory made by cabal-install , generating a report for test suite named my-project-test. Skip searching under the directories with base name my-project, and exclude modules named Main and Paths_my_project from the report. Note the use of comma to separate multiple values for the -x option:

$ hpc-codecov -X my-project -x Main,Paths_my_project cabal:my-project-test

Project using stack

Search under directory made by stack for test suite named my-project-test, show verbose information, and write output to codecov.json:

$ hpc-codecov --verbose -o codecov.json stack:my-project-test

Project using stack, with multiple packages

Search under directory made by stack for combined report of multiple cabal packages, and write output to codecov.json:

$ hpc-codecov stack:all -o codecov.json

Low-level examples

The following shows two examples for generating a test coverage report of the hpc-codecov package itself without specifying the build tool. One with using the build artifacts made by cabal-install Nix-style local build commands, and another with stack.

With cabal-install

First, run the tests with coverage option to generate .tix and mix files:

$ cabal --version
cabal-install version 3.4.0.0
compiled using version 3.4.0.0 of the Cabal library
$ cabal v2-configure --enable-test --enable-coverage
$ cabal v2-test

Then generate a Codecov JSON coverage data from the .tix and .mix files:

$ proj=hpc-codecov-0.3.0.0
$ tix=$(find ./dist-newstyle -name $proj.tix)
$ mix=$(find ./dist-newstyle -name vanilla -print -quit)/mix/$proj
$ hpc-codecov --mix=$mix --exclude=Paths_hpc_codecov --out=codecov.json $tix

The --out option specifies the output file to write the JSON report. Observing the contents of codecov.json with jq:

$ jq . codecov.json | head -10
{
  "coverage": {
    "src/Trace/Hpc/Codecov/Options.hs": {
      "48": 1,
      "50": 1,
      "52": 1,
      "54": 1,
      "56": 1,
      "59": 1,
      "63": 1,

Send the resulting JSON report file to Codecov with the bash uploader. The file name codecov.json is listed in the uploader script as one of the file name patterns to upload, no need to specify the report filename explicitly:

$ bash <(curl -s https://codecov.io/bash)

According to the Codecov FAQ, the uploader should work from Travis, CircleCI, Azure, and GitHub Actions for public projects without the Codecov token.

With stack

Build the package and run the tests with coverage option:

$ stack --numeric-version
2.5.1
$ stack build --test --coverage

As done in cabal-install example, specify the path of .tix and .mix files. Using path sub-command to get the local hpc root directory and dist directory:

$ hpcroot=$(stack path --local-hpc-root)
$ tix=$(find $hpcroot -name 'test-main.tix')
$ mix=$(stack path --dist-dir)/hpc
$ hpc-codecov --mix=$mix --exclude=Paths_hpc_codecov -o codecov.json $tix

Then send the resulting report file:

$ bash <(curl -s https://codecov.io/bash)

References

• HPC publication
• Codecov API reference