Exclude Specifically Tagged tests from SBT when using ScalaTest

Question

I'm having a lot of frustration trying to run all my tests from sbt- while excluding a specific tag. Here is what I am trying to run:

testOnly * -- -l "com.my.project.path.tags.ValidationTest"

I've tried many variations of this command, including replacing the * with the path to a class or the path to a package like so:

testOnly "com.my.project.path.somePackage" -- -l "com.my.project.path.tags.ValidationTest"

And I've tried with and without quotes around the package.

I just read that testOnly is used in the new version of sbt and not test-only. I've tried this syntax (and many variations), and nothing seems to work. I have my tests set up like this:

"some method" should "fail when doing something" taggedAs ValidationTest in { ... }

I have object ValidationTest extends Tag("com.my.project.path.tags.ValidationTest") defined in TestTag.scala.

I have also attempted this when the method is defined with the it keyword, instead of "some method" should "fail..."

Sbt with ScalaTest shows the following under "Include and Exclude Tests with Tags"

> test-only org.acme.* -- -n CheckinTests
> test-only org.acme.* -- -n FunctionalTests -l org.scalatest.tags.Slow
> test-only org.acme.* -- -n "CheckinTests FunctionalTests" -l "org.scalatest.tags.Slow org.scalatest.tags.Network"

Sheila

To be clear, you do or do not want those tags to run? Sounds like you do not want things with that tag to run, which I don't believe ScalaTest supports. – Berthaberthe

ScalaTest absolutely does support skipping these tags, or including them via the -n or -l flags. See here: ScalaTest with Sbt. See 3/4 down on the page. – Sheila

I know it is an old post, but I tried to reproduce your issue, and I can't reproduce it. Are you still experiencing this issue? – Gobioid

Possible duplicate of: Manually exclude some test classes in sbt – Gobioid

Answer 1

Just a couple of guesses:

• all this syntax works only from sbt console and DOES NOT work from command line;
• the package name should be WITHOUT quotes and should end with asterix, the tag should be WITH quotes (e.g. testOnly com.my.project.* -- -l "com.my.project.path.tags.ValidationTest")
• testOnly concerns only unit tests, if you should have integration tests, you should use e.g. it:testOnly * -- -l "path.to.Tag".

Mapel

I haven't tested this - but yes I think it has to do with me not using the sbt console - and also not using quotes – Sheila

So how can I do any of the same things from the command line? – Sheila

One thing to try out would be passing commands as parameters to sbt console e.g. in a batch file/shell script. I haven't tried it yet tho and do not know if it is at all possible. – Mapel

Answer 2

To run all test apart from tests with a specific tag:

sbt "testOnly -- -l com.my.project.path.tags.ValidationTest"

To then do the opposite, only tests with a tag:

sbt "testOnly -- -n com.my.project.path.tags.ValidationTest"

Benghazi