Karma-Coverage Report Shows Code As Covered (which is obviously not covered)

Question

I a trying to generate an HTML coverage report, but it does not contain the output I expect. Maybe I'm wrong here, but it should show only those lines and methods covered which are called from the spec file, right?

Somehow it does not.

Update:

I created an repository to provide a working example, outlining the problem:

https://github.com/gearsdigital/stunning-octo-train

This is my (test) project setup. I'm able to push it to a GitHub repo if needed as I don't know how to setup JSFiddle to run this code.

TL;DR

There is a process to generate an HTML coverage report. This report shows code as covered which is obviously not covered because there is no test available.

karma.conf.js:

var webpack = require('webpack');
var path = require('path');

// Reference webpack.config.js, don't repeat it!
var webpackConfig = require('./webpack.config.js');

// The entry point from the referenced Webpack configuration has to be
// removed or tests will fail in weird and inscrutable ways.
// Easy enough, just define an empty entry object (null won't work).
webpackConfig.entry = {};
webpackConfig.module = {
    preLoaders: [
        {
            test: /\.js$/,
            // files within these directories should be excluded
            // for babel processing
            exclude: /(node_modules)/,
            loaders: ['babel?cacheDirectory']
        },
        {
            test: /\.js$/,
            include: /(src\/js)/,
            exclude: /(vendor)/,
            loaders: ['isparta']
        }
    ]
};

/**
 * Karma configuration
 * @param config
 */
module.exports = function (config) {
    config.set({
        browsers: ['PhantomJS'],
        coverageReporter: {
            dir: 'test-results',
            reporters: [
                {type: 'text-summary'},
                {type: 'html', subdir: 'coverage'}
            ]
        },
        files: [
            'webpack.test.config.js'
        ],
        frameworks: [
            'jasmine'
        ],
        preprocessors: {
            'webpack.test.config.js': ['webpack']
        },
        reporters: ['spec', 'coverage'],
        webpack: webpackConfig
    });
};

webpack.config.js:

var webpack = require('webpack');
var path = require('path');

module.exports = {
    plugins: [
        new webpack.ProvidePlugin({
            $: "jquery",
            jQuery: "jquery",
            "window.jQuery": "jquery"
        })
    ]
};

webpack.test.config.js:

// make sure the file name regexp matches your test files.
var testsContext = require.context('./tests', true, /\.spec\.js$/);
testsContext.keys().forEach(testsContext);

// make sure the file name regexp matches your test files.
var srcContext = require.context('./src/js', true, /\.js$/);
srcContext.keys().forEach(srcContext);

bootstrap.js:

import {Calculator} from './modules/Calculator';
let c = new Calculator();
c.add(1,2); // 3

Calculator.js:

export class Calculator {
    add(op1, op2) {
        return op1 + op2;
    }

    sub(op1, op2) {
        if (typeof op1 !== 'number') {
            return false;
        }
        return op1 - op2;
    }

    mul(op1, op2) {
        return op1 * op2;
    }

    div(op1, op2) {
        return op1 / op2;
    }
}

bootstrap.spec.js:

import {Calculator} from '../src/js/modules/Calculator';

describe('Calculator', function () {

    it('should return 10', function () {
        expect(true).toBe(false);
    });

});

Generated report:

I expect add() to be uncovered, as it's not called in any test but in bootstrap.js.

Project structure:

Diallage

I don't understand why more coverage is bad :) – Morville

More coverage is not bad :) It is bad that code gets covered although there was no test written for this piece of code. – Diallage

where do you access this file with the code blocks missing coverage? – Epimenides

Answer 1

src/js/bootstrap.js gets loaded; which means these lines get executed.

import {Calculator} from './modules/Calculator';
let c = new Calculator();
c.add(1,2); // 3

I am guessing this block is the culprit:

    {
        test: /\.js$/,
        include: /(src\/js)/,
        exclude: /(vendor)/,
        loaders: ['isparta']
    }

Shouldn't my code be only exercised during tests?

The short answer is no.

Anything that is not located inside closures / function / methods may get covered when you import the module.

// this will get coverage; because it is top-level code outside a function block
var dummy = null;

export class Calculator {
    add(op1, op2) {
        // this will get coverage only if you call explicitly the function.
        return op1 + op2;
    }
}

// this will get coverage.
var c = new Calculator();
// and now, Calculator.add will get coverage.
c.add(1,2); // 3

Morville

Well, maybe i'm on the wrong track! Thank you for answering. – Diallage

Answer 2

Okay so I'm not familiar with Webpack or Babel, but I strongly suspect that since you include all your files in Karma configuration, code get executed no matter what's in the tests. I've checked out your GitHub's example (kudos on doing this btw, it's easier to debug), and if I comment out the lines 3 and 4 of cart.js, the coverage report doesn't show utilities.js as 'ran'.

Long story short : code that's not encapsulated into a function gets run by including the files in Karma, obviously. Also, don't trust Karma when there's no tests at all (like, literally 0 it), it's unreliable in this specific case.

Stephaniastephanie

Thanks for your answer! Do you have any suggestions how to deal with that issue? I thought to have some encapsulation as im using ES2015 Modules. – Diallage

Well, yeah, you just shoudn't have calls to functions at the top of your JS files. Because what happens, really, is that when included, the file cart.js calls the method objectLength(). Karma "sees" the code as called during testing and therefore marks it as covered. – Stephaniastephanie