Can I write 'parameterized' tests in DUnit

Question

I am using DUnit to test a Delphi library. I sometimes run into cases, where i write several very similar tests to check multiple inputs to a function.

Is there a way to write (something resembling) a parameterized test in DUnit? For instance specifying an input and expected output to a suitable test procedure, then running the test suite and getting feedback on which of the multiple runs of the test failed?

(Edit: an example)

For example, suppose I had two tests like this:

procedure TestMyCode_WithInput2_Returns4();
var
  Sut: TMyClass;
  Result: Integer;
begin
  // Arrange:
  Sut := TMyClass.Create;

  // Act:
  Result := sut.DoStuff(2);

  // Assert
  CheckEquals(4, Result);
end;

procedure TestMyCode_WithInput3_Returns9();
var
  Sut: TMyClass;
  Result: Integer;
begin
  // Arrange:
  Sut := TMyClass.Create;

  // Act:
  Result := sut.DoStuff(3);

  // Assert
  CheckEquals(9, Result);
end;

I might have even more of these tests that do exactly the same thing but with different inputs and expectations. I don't want to merge them into one test, because I would like them to be able to pass or fail independently.

Kickback

Do you mean dynamic creation of test cases for all input values in a list? My (small) OpenCTF test framework contains code for dynamic creation of test cases. It is based on DUnit. – Acquit

You can always write a general parameterized method in the testclass and call that from one or more specific (published) test methods. The Check(Not)Equals method(s) of a TestCase can help here as well to help keep code concise and still provide a specific failure message for each test. – Fronton

@Marjan the test method will stop execution as soon as the first Check(Not)Equals fails - dynamic creation of test cases solves this problem, all other values will still be tested – Acquit

@Acquit : OpenCTF seems to be for testing components and forms in a black box manner... That does not seem applicable here... – Kickback

@MarjanVenema : That's not a bad way to do it I guess. I'll try that one out... – Kickback

@MathiasFalkenberg: can you be more concrete about what kind of tests you are trying to write? I am now uncertain of whether you want to exercise the same function with multiple inputs or whether you want to check the inputs when in a function? In other words: please be more specific about the testcases you want to run so we can be more to the point with possibilites. (@Acquit and I at least seem to interpret it differently). – Fronton

@MarjanVenema : I've tried adding an example. What I would like to do is exercise the same functionality with different inputs and then check the result. – Kickback

Answer 1

You can use DSharp to improve your DUnit tests. Especially the new unit DSharp.Testing.DUnit.pas (in Delphi 2010 and higher).

Just add it to your uses after TestFramework and you can add attributes to your test case. Then it could look like this:

unit MyClassTests;

interface

uses
  MyClass,
  TestFramework,
  DSharp.Testing.DUnit;

type
  TMyClassTest = class(TTestCase)
  private
    FSut: TMyClass;
  protected
    procedure SetUp; override;
    procedure TearDown; override;
  published
    [TestCase('2;4')]
    [TestCase('3;9')]
    procedure TestDoStuff(Input, Output: Integer);
  end;

implementation

procedure TMyClassTest.SetUp;
begin
  inherited;
  FSut := TMyClass.Create;
end;

procedure TMyClassTest.TearDown;
begin
  inherited;
  FSut.Free;
end;

procedure TMyClassTest.TestDoStuff(Input, Output: Integer);
begin
  CheckEquals(Output, FSut.DoStuff(Input));
end;

initialization
  RegisterTest(TMyClassTest.Suite);

end.

When you run it your test looks like this:

Since attributes in Delphi just accept constants the attributes just take the arguments as a string where the values are separated by a semicolon. But nothing prevents you from creating your own attribute classes that take multiple arguments of the correct type to prevent "magic" strings. Anyway you are limited to types that can be const.

You can also specify the Values attribute on each argument of the method and it gets called with any possible combination (as in NUnit).

Referring to the other answers personally I want to write as little code as possible when writing unit tests. Also I want to see what the tests do when I look at the interface part without digging through the implementation part (I am not going to say: "let's do BDD"). That is why I prefer the declarative way.

Markusmarl

+1 That looks very useful and interesting indeed. Thanks for bringing it to our attention. – Glanville

+1 Indeed! I'll be sure to look into that. I think this is the most hassle-free approach suggested so far! – Kickback

Answer 2

I think you are looking for something like this:

unit TestCases;

interface

uses
  SysUtils, TestFramework, TestExtensions;

implementation

type
  TArithmeticTest = class(TTestCase)
  private
    FOp1, FOp2, FSum: Integer;
    constructor Create(const MethodName: string; Op1, Op2, Sum: Integer);
  public
    class function CreateTest(Op1, Op2, Sum: Integer): ITestSuite;
  published
    procedure TestAddition;
    procedure TestSubtraction;
  end;

{ TArithmeticTest }

class function TArithmeticTest.CreateTest(Op1, Op2, Sum: Integer): ITestSuite;
var
  i: Integer;
  Test: TArithmeticTest;
  MethodEnumerator: TMethodEnumerator;
  MethodName: string;
begin
  Result := TTestSuite.Create(Format('%d + %d = %d', [Op1, Op2, Sum]));
  MethodEnumerator := TMethodEnumerator.Create(Self);
  Try
    for i := 0 to MethodEnumerator.MethodCount-1 do begin
      MethodName := MethodEnumerator.NameOfMethod[i];
      Test := TArithmeticTest.Create(MethodName, Op1, Op2, Sum);
      Result.addTest(Test as ITest);
    end;
  Finally
    MethodEnumerator.Free;
  End;
end;

constructor TArithmeticTest.Create(const MethodName: string; Op1, Op2, Sum: Integer);
begin
  inherited Create(MethodName);
  FOp1 := Op1;
  FOp2 := Op2;
  FSum := Sum;
end;

procedure TArithmeticTest.TestAddition;
begin
  CheckEquals(FOp1+FOp2, FSum);
  CheckEquals(FOp2+FOp1, FSum);
end;

procedure TArithmeticTest.TestSubtraction;
begin
  CheckEquals(FSum-FOp1, FOp2);
  CheckEquals(FSum-FOp2, FOp1);
end;

function UnitTests: ITestSuite;
begin
  Result := TTestSuite.Create('Addition/subtraction tests');
  Result.AddTest(TArithmeticTest.CreateTest(1, 2, 3));
  Result.AddTest(TArithmeticTest.CreateTest(6, 9, 15));
  Result.AddTest(TArithmeticTest.CreateTest(-3, 12, 9));
  Result.AddTest(TArithmeticTest.CreateTest(4, -9, -5));
end;

initialization
  RegisterTest('My Test cases', UnitTests);

end.

which looks like this in the GUI test runner:

I'd be very interested to know if I have gone about this in a sub-optimal way. DUnit is so incredibly general and flexible that whenever I use it I always end up feeling that I've missed a better, simpler way to solve the problem.

Glanville

I feel the same way... Which is why I posted this question. While your code certainly produces the desired output, I would like my tests to be more readable. The 'CreateTest' method introduces a layer of complexity to the test code that I'd really rather avoid... – Kickback

Answer 3

Would it be sufficient if DUnit allowed to write code like this, where every call of AddTestForDoStuff would create a test case similar to those in your example?

Suite.AddTestForDoStuff.With(2).Expect(4);
Suite.AddTestForDoStuff.With(3).Expect(9);

I'll try to post an example how this can be done later today...

---

For .Net there is already something similar: Fluent Assertions

http://www.codeproject.com/Articles/784791/Introduction-to-Unit-Testing-with-MS-tests-NUnit-a

Acquit

Something along those lines would be nice. But it would probably need to take the specific test as an argument also, right? Such as Suite.AddTest('DoStuff').WithArgument(2).Expects(4) – Kickback

@MathiasFalkenberg: That or at least the possibility to add a message. – Fronton

This answer is quite cool. It's the first time I've ever seen an up-vote awarded for wishful thinking. Yeah, wouldn't it be just great if you could do this!! Anyway, the +1 down-payment will be earned if you can produce code that actually follows through. – Glanville

Hint: AddTestForDoStuff creates a TTestDoStuff instance, With and Expect work like property setters. It is the builder pattern, applied to DUnit. – Acquit

Answer 4

Here is an example of using a general parameterized test method called from your TTestCase descendants actual (published) test methods (:

procedure TTester.CreatedWithoutDisplayFactorAndDisplayString;
begin
  MySource := TMyClass.Create(cfSum);

  SendAndReceive;
  CheckDestinationAgainstSource;
end;

procedure TTester.CreatedWithDisplayFactorWithoutDisplayString;
begin
  MySource := TMyClass.Create(cfSubtract, 10);

  SendAndReceive;
  CheckDestinationAgainstSource;
end;

Yes, there is some duplication, but the main duplication of code was taken out of these methods into the SendAndReceive and CheckDestinationAgainstSource methods in an ancestor class:

procedure TCustomTester.SendAndReceive;
begin
  MySourceBroker.CalculationObject := MySource;
  MySourceBroker.SendToProtocol(MyProtocol);
  Check(MyStream.Size > 0, 'Stream does not contain xml data');
  MyStream.Position := 0;
  MyDestinationBroker.CalculationObject := MyDestination;
  MyDestinationBroker.ReceiveFromProtocol(MyProtocol);
end;

procedure TCustomTester.CheckDestinationAgainstSource(const aCodedFunction: string = '');
var
  ok: Boolean;
  msg: string;
begin
  if aCodedFunction = '' then
    msg := 'Calculation does not match: '
  else
    msg := 'Calculation does not match. Testing CodedFunction ' + aCodedFunction + ': ';

  ok := MyDestination.IsEqual(MySource, MyErrors);
  Check(Ok, msg + MyErrors.Text);
end;

The parameter in the CheckDestinationAgainstSource also allows for this type of use:

procedure TAllTester.AllFunctions;
var
  CF: TCodedFunction;
begin
  for CF := Low(TCodedFunction) to High(TCodedFunction) do
  begin
    TearDown;
    SetUp;
    MySource := TMyClass.Create(CF);
    SendAndReceive;
    CheckDestinationAgainstSource(ConfiguredFunctionToString(CF));
  end;
end;

This last test could also be coded using the TRepeatedTest class, but I find that class rather unintuitive to use. The above code gives me greater flexibility in coding checks and producing intelligible failure messages. It does however have the drawback of stopping the test on the first failure.

Fronton

Answer 5

At least in Delphi XE8 just out the box coming a DUnitX with a similar attribute:

[TestCase('Test1', '1,one')]
[TestCase('Test2', '2,two')]
procedure TestIntToStr(AValInt: Integer; const AValString: string);

(See also https://docwiki.embarcadero.com/RADStudio/Seattle/en/DUnitX_Overview and https://web.archive.org/web/20150504232249/http://docwiki.embarcadero.com/RADStudio/XE8/en/DUnitX_Overview)

Also it available for Delphi from version 2010 at https://github.com/VSoftTechnologies/DUnitX

Viewfinder