Cannot get code coverage with Cypress + Istanbul

Question

I tried setting up code coverage in an Angular 8 project using Cypress and istanbul nyc.

I managed to get the code instrumented (the global variable __coverage__ is properly set) :

and the coverage file generated in .nyc_output after running cypress:open

But the generated coverage report is empty:

$ cat coverage/coverage-final.json
{}

Same result when I execute the command:

$ npx nyc report --report-dir ./coverage --temp-dir .nyc_output --reporter=text
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|

Here is my package.json devDependencies:

"devDependencies": {
  "@angular-devkit/build-angular": "^0.803.3",
  "@angular-devkit/build-optimizer": "^0.803.3",
  "@angular/cli": "^8.3.3",
  "@angular/compiler-cli": "8.2.5",
  "@angular/language-service": "8.2.5",
  "@briebug/cypress-schematic": "^2.0.0",
  "@cypress/code-coverage": "^1.10.1",
  "@cypress/webpack-preprocessor": "^4.1.0",
  "@istanbuljs/nyc-config-typescript": "^0.1.3",
  "@types/jasmine": "^3.4.0",
  "@types/jasminewd2": "^2.0.6",
  "@types/node": "^12.7.4",
  "babel-plugin-istanbul": "^5.2.0",
  "codelyzer": "^5.1.0",
  "cypress": "^3.4.1",
  "istanbul-instrumenter-loader": "^3.0.1",
  "istanbul-lib-coverage": "^2.0.5",
  "jasmine-core": "^3.4.0",
  "jasmine-spec-reporter": "4.2.1",
  "karma": "^4.3.0",
  "karma-chrome-launcher": "^3.1.0",
  "karma-cli": "^2.0.0",
  "karma-coverage-istanbul-reporter": "^2.1.0",
  "karma-jasmine": "^2.0.1",
  "karma-jasmine-html-reporter": "^1.4.2",
  "mochawesome": "^4.1.0",
  "ngx-build-plus": "^8.1.4",
  "nyc": "^14.1.1",
  "protractor": "^5.4.2",
  "protractor-html-reporter-2": "^1.0.4",
  "protractor-http-client": "^1.0.4",
  "source-map-support": "^0.5.13",
  "ts-node": "^8.3.0",
  "tslib": "^1.10.0",
  "tslint": "^5.19.0",
  "typescript": "3.5.3"
}

And my .nycrc.json:

{
    "cache": false,
    "extension": [
      ".ts",
      ".tsx"
    ],
    "exclude": [
      "**/*.d.ts",
      "coverage/**",
      "packages/*/test/**",
      "test/**",
      "test{,-*}.ts",
      "**/*{.,-}{test,spec}.ts",
      "**/__tests__/**",
      "**/node_modules/**"
    ],
    "all": true,
    "check-coverage": true,
    "require": [
      "ts-node/register"
    ],
    "temp-directory": ".nyc_output",
    "sourceMap": false,
    "instrument": false,
    "include": ["src/**/*.ts", "src/**/*.tsx"]
}

Heartworm

Do you have this code in Support file? // cypress/support/index.js import '@cypress/code-coverage/support' – Sekyere

@Sekyere nope sorry – Heartworm

sorry i misunderstood the question (saw it before it was edited). yes I do have the support and plugin files modified, you can see that the tasks are executed (resetCoverage, combineCoverage, coverageReport) – Heartworm

the problem is NYC not generating the correct output – Heartworm

Answer 1

It seems to be an issue in nyc when excludeAfterRemap is true.

setting it to false fixed the problem.

Heartworm

Answer 2

Here is details description step by step. https://docs.cypress.io/guides/tooling/code-coverage.html#E2E-code-coverage

I think you are missing code-coverage/support file in index.js Please do follow change and run. I think you should be good.

    // cypress/support/index.js
       import '@cypress/code-coverage/support'

   // cypress/plugins/index.js   
      module.exports = (on, config) => {
      on('task', require('@cypress/code-coverage/task'))}

When you run the Cypress tests now, you should see a few commands after the tests finish. We have highlighted these commands using a green rectangle below.

Sekyere

thanks for the reply, I do however have the commands as you can see in the question (first screenshot) – Heartworm

Another thing I missed which is necessary is - return the config. docs.cypress.io/guides/tooling/code-coverage#Install-the-plugin module.exports = (on, config) => {require('@cypress/code-coverage/task')(on, config); /*other stuff;*/ return config;}; – Tweed

There is another hidden problem. I'm using typescript. I realised in one of my examples my first page - src/index.js was extension .js. For a day I couldn't figure out why my coverage wasn't being rendered. When I changed index.js to index.tsx the coverage was then rendred. – Tweed

Answer 3

Here is my advice:

1. First, make sure your tests are running against your local serve instance.

   Example: ng serve --configuration=demo & cypress run --headed ...

2. In my experience the .nycrc.json is not really needed.

3. The end of your package.json needs this:

"nyc": { "report-dir": "coverage-e2e" }

4. Babel is not required; not everyone uses it. Instead of Babel, a Webpack build config would also work: ./cypress/coverage.webpack.js

Finally, you should know:

You can instrument your server and then generate a report with these commands:

## first, instrument your server locally
nyc --silent ng serve

{ then manually test your app or run automation against it }

## then generate your report from the json files that were generated above
nyc report --reporter html --reporter text  --report-dir ./cov-e2e/summary

Teenager

Answer 4

With @cypress/code-coverage, you have to know that the plugin does not instrument your code. You can use instanbul for that purpose. There is for example, this plugin babel-plugin-istanbul that can help you achieve it.

if you are already using babel, you need to add the plugin to your .babelrc as follow:

{
  "plugins": ["istanbul"]
}

For more info check https://github.com/cypress-io/code-coverage#instrument-your-application

Offprint

Not true, you don't have to use babel to do this. Answer is misleading. – Teenager

Hey @djangofan, I updated my answer. Thank you for pointing it out. – Offprint