How do you impose scala code coverage specifically for integration tests?

Question

Am running the integration test using following sbt command

sbt clean coverage it:test coverageReport

This command runs integration tests, instruments it and generates report as well.

Build.sbt has following:

coverageMinimum in IntegrationTest := 21.0
coverageFailOnMinimum in IntegrationTest := true

Output looks like:

[info] Statement coverage.: 20.16%
[info] Branch coverage....: 12.00%
[info] Coverage reports completed
[info] All done. Coverage was [20.16%]

Output result has 20.16% code coverage but the limits in build.sbt are not enforcing the limit.

If I change build.sbt to following it works:

coverageMinimum := 21.0
coverageFailOnMinimum := true

Wanted to know what am I missing for specifying limits specifically for Integration tests

Version Information:

sbt : 0.13.17

sbt-scoverage : 1.5.1

Shortly

It works with the appropriate setting coverageFailOnMinimum, which happens not to be set by default, so what? – Zibet

What do you mean by appropriate setting ? Like the same two settings defined within Integration scope doesn’t seem to be recognized. What am I missing ? – Shortly

Answer 1

The following two workarounds seem to work on my machine (sbt-scoverage 1.5.1, sbt 1.1.1, scala 2.12.5)

Workaround 1 - Use inConfig to scope to a configuration:

inConfig(IntegrationTest)(ScoverageSbtPlugin.projectSettings),
inConfig(IntegrationTest)(Seq(coverageMinimum := 21, coverageFailOnMinimum := true))

Now executing sbt clean coverage it:test it:coverageReport throws Coverage minimum was not reached.

Workaround 2 - Modify coverageMinimum setting within a custom command:

def itTestWithMinCoverage = Command.command("itTestWithMinCoverage") { state =>
  val extracted = Project extract state
  val stateWithCoverage = extracted.append(Seq(coverageEnabled := true, coverageMinimum := 21.0, coverageFailOnMinimum := true), state)
  val (s1, _) = Project.extract(stateWithCoverage).runTask(test in IntegrationTest, stateWithCoverage)
  val (s2, _) = Project.extract(s1).runTask(coverageReport in IntegrationTest, s1)
  s2
}

commands ++= Seq(itTestWithMinCoverage)

Now executing sbt itTestWithMinCoverage throws Coverage minimum was not reached. Note after executing itTestWithMinCoverage the state is discarded so coverageMinimum should be back to default value, and thus not affect unit tests.

It seems the issue is (besides my lack of understanding how scopes exactly work) checkCoverage picks up default value of coverageMinimum even after setting coverageMinimum in IntegrationTest.

Herrin

I tried the inconfig, 1st approach, didn't work for me. have an older version of scala then yours. i defined for test and integration test but it doesn't seem to read it – Shortly