Doing something before or after all Scalatest tests
Asked Answered
E

3

79

I have a suite of scalatest tests that test different endpoints of a RESTful API. I really want them separated into different files for best organization.

My problem is how to start something (an HTTP server in my case, but it doesn't matter what it is) before all the tests and shut it down after all the tests are done.

I know about BeforeAndAfterAll, but that only accomplishes before/after inside one test file. I need something like that, but for all tests, for example:

-- start http server before tests
-- run all test suites
-- shut down http server

Engineer answered 15/3, 2013 at 1:45 Comment(2)
See this question / answer (not quite a duplicate) -> #8487369Ale
Check this q/a: #27273311 I wanted to keep the initialization step as a test itself, with dependent tests making sure the initialization was run first, and canceling if it had failedLundeen
E
61

The intended way to do this is to use nested suites. Suite has a nestedSuites method that returns an IndexedSeq[Suite] (in 2.0, in 1.9.1 it was a List[Suite]). Suite also has a runNestedSuites method that is responsible for executing any nested suites. By default runNestedSuites calls nestedSuites, and on each returned Suite either invokes run directly, or if a Distributor is passed, puts the nested suites in the distributor so that they can be run in parallel.

So what you really probably want to do is make Foo and Bar into classes, and return instances of them from the nestedSuites method of EndpointTests. There's a class that makes that easy called Suites. Here's an example of its use:

import org.scalatest._
import matchers.MustMatchers

class Foo extends FunSpec with MustMatchers {
  describe("Message here...") {
    it("Must do something") {  }
    it("Must be ok") {  }
  }
}

class Bar extends FunSpec with MustMatchers {
  describe("Hello you...") {
    it("One more!") {  }
  }
}

class EndpointTests extends Suites(new Foo, new Bar) with BeforeAndAfterAll {

  override def beforeAll(configMap: Map[String, Any]) {
    println("Before!")  // start up your web server or whatever
  }     

  override def afterAll(configMap: Map[String, Any]) {
    println("After!")  // shut down the web server
  }         
}

One potential problem, though, is that if you are using discovery to find Suites to run, all three of EndpointTests, Foo, and Bar will be discovered. In ScalaTest 2.0 you can annotate Foo and Bar with @DoNotDiscover, and ScalaTest's Runner will not discover them. But sbt still will. We are currently enhancing sbt so that it passes over otherwise discoverable Suites that are annotated with DoNotDiscover, but this will be in sbt 0.13, which isn't out yet. In the meantime you can get sbt to ignore them by adding an unused constructor parameter to Foo and Bar.

Endymion answered 21/3, 2013 at 19:27 Comment(6)
Playing with this a bit... Trying the unused constructor parameter idea. It works from a no-discover standpoint but throws some kind of error: java.lang.IllegalArgumentException: Class is not an accessible org.scalatest.Suite: com.br.awe.service.test.Api10tests . at org.scalatest.tools.ScalaTestFramework$ScalaTestRunner.run(ScalaTestFramework.scala:300) at org.scalatools.testing.Runner2.run(Runner2.java:16) at sbt.TestRunner.delegateRun(TestFramework.scala:57) ... My master test suite starts like this: class MasterTestSuite extends Suites(new Api10tests(1), new Api20tests(1))Engineer
Is there a way to show the results for the inner tests? At the moment, if it fails, it simply sats that the EndpointTests failed. It would be nice to see which tests failed inside of the suite.Holmquist
There's a bug that causes a deadlock for this approach in ScalaTest < 2.0.M8, so make sure you upgrade. See release notes for M8 here: scalatest.org/release_notes/2.0Stringer
In the scalatest 2.2.5 you do not need to pass config to beforeAll method: ` @deprecated("Please use the beforeAll(ConfigMap) method of trait BeforeAndAfterAllConfigMap instead.") protected def beforeAll(configMap: ConfigMap) { beforeAll() } `Swinge
Just to note, using this approach we lose ability to run individual suites.Pleurisy
@IvanBalashov what do you mean by "run individual tests"?Bellbella
E
12

Ok, found a way. It seems (unless someone here can correct me) that Scalatest does not have the facility of a "master" suite. But... you can kinda build one.

You can compose a suite from traits. So using my endpoint example:

class EndpointTests extends FunSpec with MustMatchers with BeforeAndAfterAll 
      with Foo with Bar {
        override def beforeAll(configMap: Map[String, Any]) {
            println("Before!")  // start up your web server or whatever
        }

        override def afterAll(configMap: Map[String, Any]) {
            println("After!")  // shut down the web server
        }   
}

Ok, but what about the tests? Notice the with Foo with Bar. I'm bringing the dependent tests in as traits. See here:

trait Foo extends FunSpec with MustMatchers {
    describe("Message here...") {
        it("Must do something") {  }
        it("Must be ok") {  }
    }
}

trait Bar extends FunSpec with MustMatchers { 
    describe("Hello you...") {
        it("One more!") {  }
    }
}
Engineer answered 15/3, 2013 at 2:31 Comment(1)
Bill's answer above suffers from test failure details being swallowed - which is useless if you actually want to know what went wrong. Unfortunately this approach has the side effect that BeforeAndAfterAll even within individual tests applies to the entire suite. Also that you no longer can run tests individually, and that their output is written all in one go to the console. I really wish ScalaTest had a better answer than this.Walz
B
11

Alternatively you can just use an object.

object TestServer {
  startServer()
}

When you access the object it will be initialised, starting the server. Just create a common trait in the body of which you access the object. Then mixin that trait into all your tests. Done.

If your server runs in daemon mode (e.g. a Play! application in test mode) it will be automatically shut down after all tests are run.

Beachhead answered 1/6, 2014 at 16:31 Comment(2)
I can see this working in many cases better for me, than 'BeforeAndAfterAll'. e.g. now I'm testing with an etcd back-end, and using an Object allows me to have the same instance running, for multiple tests.Lundeen
object TestServer { startServer(); override def finalize(): Unit = { super.finalize(); stopServer() }}Revulsion

© 2022 - 2024 — McMap. All rights reserved.