Compile-time Call Count in Nim

Question

The following code does not compile but illustrates what I would like to do: totalTests should hold the number of time that assertEquals() is called (assertEquals() should probably be a macro for this to be possible, but I'm not familiar with this aspect of Nim yet).

Any idea how this code should be modified to enable the following code to print [1/2] and [2/2] at the beginning of each test report line?

from strutils import format

var countTested = 0
var countFailed = 0
var countPassed = 0
let totalTests = 0 # <-- need let or other compile-time type here (?)

# using proc here... macro may be needed to be able to count calls (?)
proc assertEquals*[T](testName: string, expected: T, p: (proc(): T)) =
   countTested += 1
   totalTests += 1 # <-- compilation error (how can we increase each time it is called?)
   write(stdout, format("[$num/$total] $name: ", "num", countTested, "total", totalTests, "name", testName))
   var val = p()
   if val == expected:
     write(stdout, "passed\n")
     countPassed += 1
   else:
     write(stdout, "failed\n")
     countFailed += 1

when isMainModule:
  assertEquals("test #A", 12, proc(): int = 14-2)
  assertEquals("test #B", 12, proc(): int = 12-2)

Edit: added questions in code

Glorianna

var totalTests instead of let. Let is probably immutable. I know nothing of this language. – Infrequency

You are right luqui, here I used let to illustrate the intent (compile time calculation, Nim seems to be good at performing compile-time operations, so there might be a way to do this). But indeed the example does not compile. – Glorianna

By the way, have you noticed the unittest module (nim-lang.org/docs/unittest.html)? One interesting insight about Nim is that it doesn't need the large suite of assert functions used in other languages (assertEquals, assertGreater, etc). The reason for this is that you can write a single assert macro that will examine the passed expression and recognize the used operator (==, <, etc) to print out the correct assertion failed message. In the unittest module, this is the check macro. – Imperturbable

Thanks zah. Actually I started this to get some practice on Nim and found out after this about unittest. unittest is more powerful than the little program I posted below. This example of usage of unittest helped me: github.com/Araq/Nim/blob/master/examples/tunit.nim – Glorianna

Answer 1

Here is one way to do it. You can execute code at compile time by using a macro or a static statement. Note that there's still no way to reliably count these across multiple modules.

import macros, strutils

proc beginTests()

var countTested = 0
var countFailed = 0
var countPassed = 0
var totalTests = 0
var totalTestsCT {.compiletime.} = 0

macro endTests(): stmt =
  quote do:
    proc beginTests() =
      totalTests = `totalTestsCT`

proc assertEqualsImpl*[T](testName: string, expected: T, p: (proc(): T)) =
   countTested += 1
   write(stdout, format("[$num/$total] $name: ",
         "num", countTested, "total", totalTests, "name", testName))
   var val = p()
   if val == expected:
     write(stdout, "passed\n")
     countPassed += 1
   else:
     write(stdout, "failed\n")
     countFailed += 1

macro assertEquals*[T](testName: string, expected: T, p: (proc(): T)): stmt =
  totalTestsCT += 1
  quote do:
    assertEqualsImpl(`testName`, `expected`, `p`)

when isMainModule:
  beginTests()
  assertEquals("test #A", 12, proc(): int = 14-2)
  assertEquals("test #B", 12, proc(): int = 12-2)
  endTests()

An alternative implementation would be to embed the tests in a custom block statement, e.g.

testSuite:
  assertEquals("test #A", 12, proc(): int = 14-2)
  assertEquals("test #B", 12, proc(): int = 12-2)

The testSuite macro would then count the assertions in the embedded code and initialize the variable accordingly.

Yet another solution would be to not execute the tests directly, but store them in a list and only execute them at the end.

Rudin

Thanks Reimer. Your last suggestion worked best for me. I edited your answer to add the code for that one. Your macro-based solution is nice and to the point, but it appears macros are probably not the best way to do this. Especially, it did not work when called from another module (as you mention in your answer). In this case the compiler complains about beginTests() not being implemented, even if endTests() is called in that module. Anyway, thanks for your answer! – Glorianna

Well actually my edit was rejected, so the implementation is added as a separate answer... – Glorianna

For what it's worth, I didn't even see the edit before it was rejected, and it looks fine to me. – Rudin

Answer 2

Here is an implementation of Reimer's third suggestion, which worked best for me.

import macros, strutils

type 
  TestSuiteObj = object
    countTested: int
    countFailed: int
    countPassed: int
    totalTests: int
    tests: seq[(proc (self: TestSuite))]
  TestSuite* = ref TestSuiteObj


proc newTestSuite*(): TestSuite =
  new(result)
  result.countTested = 0
  result.countFailed = 0
  result.countPassed = 0
  result.totalTests = 0
  result.tests = @[]

proc assertEquals*[T](self: TestSuite, testName: string, expected: T, p: (proc(): T)) =
  self.totalTests += 1

  var testProc = proc(self: TestSuite) =
    self.countTested += 1
    write(stdout, format("[$num/$total] $name: ", "num", self.countTested, "total", self.totalTests, "name", testName))
    var val = p()
    if val == expected:
      write(stdout, "passed\n")
      self.countPassed += 1
    else:
      write(stdout, "failed\n")
      self.countFailed += 1

  self.tests.add(testProc)


proc run*(self: TestSuite) =
  self.totalTests = self.tests.len
  for p in self.tests:
    p(self)

  var verdict = case (self.countTested == self.countPassed)
  of true: "PASSED"
  of false: "FAILED"
  echo format("$verdict. Passed [$passed/$total] tests.", "verdict", verdict, "passed", self.countPassed, "total", self.countTested)

  # Sanity
  assert(self.countTested == (self.countFailed+self.countPassed))
  assert(self.countTested == self.totalTests)


when isMainModule:
  var suite = newTestSuite() 
  suite.assertEquals("test #A", 12, proc(): int = 14-2)
  suite.assertEquals("test #B", 12, proc(): int = 12-2)
  suite.run()

Glorianna