How to show custom failure messages in ScalaTest?

Question

Does anyone know how to show a custom failure message in ScalaTest?

For example:

NumberOfElements() should equal (5)

Shows the following message when it fails:

10 did not equal 5

But i want more descriptive message like:

NumberOfElements should be 5.

Emotionalism

Answer 1

You're the first to ask for such a feature. One way to achieve this is with withClue. Something like:

withClue("NumberOfElements: ") { NumberOfElements() should be (5) }

That should get you this error message:

NumberOfElements: 10 was not equal to 5

If you want to control the message completely you can write a custom matcher. Or you could use an assertion, like this:

assert(NumberOfElements() == 5, "NumberOfElements should be 5")

Can you elaborate on what your use case is? Why is it that 10 did not equal 5 is not up to snuff, and how often have you had this need?

Here's the kind of thing you're requesting:

scala> import org.scalatest.matchers.ShouldMatchers._
import org.scalatest.matchers.ShouldMatchers._

scala> withClue ("Hi:") { 1 + 1 should equal (3) }
org.scalatest.TestFailedException: Hi: 2 did not equal 3
at org.scalatest.matchers.Matchers$class.newTestFailedException(Matchers.scala:150)
at org.scalatest.matchers.ShouldMatchers$.newTestFailedException(ShouldMatchers.scala:2331)


scala> class AssertionHolder(f: => Any) {
     |   def withMessage(s: String) {
     |     withClue(s) { f }
     |   }
     | }
defined class AssertionHolder

scala> implicit def convertAssertion(f: => Any) = new AssertionHolder(f)
convertAssertion: (f: => Any)AssertionHolder

scala> { 1 + 1 should equal (3) } withMessage ("Ho:")
org.scalatest.TestFailedException: Ho: 2 did not equal 3
at org.scalatest.matchers.Matchers$class.newTestFailedException(Matchers.scala:150)
at org.scalatest.matchers.ShouldMatchers$.newTestFailedException(ShouldMatchers.scala:2331)

So this way you can write:

{ NumberOfElements() should be (5) } withMessage ("NumberOfElements:")

Mazel

There are situations where i had to put more than one assertion in a it() test and there are more than one integer comparision. It is not clear by looking at the logs which assertion has failed. – Emotionalism

But the withClue way of specifying it is not readable. Isnt there a way of specifying the message at the end? – Emotionalism

At the end isn't doable with the matcher's DSL, but you can write a method that puts the withClue params in the opposite order. I'll add an example to the answer. – Mazel

Answer 2

New way since 2011: Matchers and AppendedClue¹ traits. Also, for collection sizes, there are some default messages.

import org.scalatest.{AppendedClues, Matchers, WordSpec}

class SomeTest extends WordSpec with Matchers with AppendedClues {

  "Clues" should {
    "not be appended" when {
      "assertions pass" in {
        "hi" should equal ("hi") withClue "Greetings scala tester!"
      }
    }
    "be appended" when {
      "assertions fail"  in {
        1 + 1 should equal (3) withClue ", not even for large values of 1!"
      }
    }
    "not be needed" when {
      "looking at collection sizes" in {
        val list = List(1, 2, 3)
        list should have size 5
      }
    }
  }
}

Output looks like this:

SomeTest:
Clues
  should not be appended
  - when assertions pass
  should be appended
  - when assertions fail *** FAILED ***
    2 did not equal 3, not even for large values of 1! (SomeTest.scala:15)
  should not be needed
  - when looking at collection sizes *** FAILED ***
    List(1, 2, 3) had size 3 instead of expected size 5 (SomeTest.scala:21)

Note that the List size message isn't great for lists with long .toString output.

See the scaladoc for more information.

---

¹ I'm guessing the AppendedClues trait was inspired by this question, Bill Venners of the accepted answer is the author of this trait.

Schmuck

Answer 3

You can also use withClue without importing anything or adding it to the test class:

withClue(s"Expecting distinct elements: ${elements.toList}") { elements.length shouldBe 3 }

This is imported from Assertions class: org.scalatest.Assertions#withClue

Desjardins

What does this add on top of the accepted answer? – Schmuck

Answer 4

Few things that none of the existing answers mention.

1. The position of withClue determines where the custom message goes. If written as withClue(<message>) { <assertion> }, the message is prepended, if written as { <assertion> } withClue(<message>), the message is appended.
2. In order to use withClue as an infix operator, org.scalatest.AppendedClues must be mixed in, or org.scalatest.AppendedClues.convertToClueful imported.
3. Scala 3 doesn't allow methods with entirely alphabetic names to be used as infix if not defined with the infix keyword. In order to get around this, enclose withClue in backticks, like so: { <assertion> } `withClue` (<message>)

Excitant