Can Googletest value-parameterized with multiple, different types of parameters match mbUnit flexibility?
Asked Answered
S

2

28

I'd like to write C++ Google tests which can use value-parameterized tests with multiple parameters of different data types, ideally matching the complexity of the following mbUnit tests written in C++/CLI.

For an explanation of mbUnit, see the Hanselman 2006 article. As of this 2019 edit, the other links he includes are dead.

Note how compact this is, with the [Test] attribute indicating this is a test method and the [Row(...)] attributes defining the values for an instantiation.

[Test]
[Row("Empty.mdb", "select count(*) from collar", 0)]
[Row("SomeCollars.mdb", "select count(*) from collar", 17)]
[Row("SomeCollars.mdb", "select count(*) from collar where max_depth=100", 4)]
void CountViaDirectSQLCommand(String^ dbname, String^ command, int numRecs)
{
   String^ dbFilePath = testDBFullPath(dbname);
   {
       StAnsi fpath(dbFilePath);
       StGdbConnection db( fpath );
       db->Connect(fpath);
       int result = db->ExecuteSQLReturningScalar(StAnsi(command));
       Assert::AreEqual(numRecs, result);
   }
}

Or even better, this more exotic testing from C# (pushing the boundaries of what can be defined in .Net attributes beyond what's possible in C++/CLI):

[Test]
[Row("SomeCollars.mdb", "update collar set x=0.003 where hole_id='WD004'", "WD004",
    new string[] { "x", "y" },
    new double[] { 0.003, 7362.082 })]  // y value unchanged 
[Row("SomeCollars.mdb", "update collar set x=1724.8, y=6000 where hole_id='WD004'", "WD004",
    new string[] { "x", "y" },
    new double[] { 1724.8, 6000.0 })]
public void UpdateSingleRowByKey(string dbname, string command, string idValue, string[] fields, double[] values)
{
...
}

The help says Value-parameterized tests will let you write your test only once and then easily instantiate and run it with an arbitrary number of parameter values. but I'm fairly certain that is referring to the number of test cases.

Even without varying the data types, it seems to me that a parameterized test can only take one parameter?

2019 update

Added because I got pinged about this question. The Row attribute shown is part of mbUnit.

For an explanation of mbUnit, see the Hanselman 2006 article. As of this 2019 edit, the other links he includes are dead.

In the C# world, NUnit added parameterised testing in a more powerful and flexible way including a way to handle generics as Parameterised Fixtures.

The following test will be executed fifteen times, three times for each value of x, each combined with 5 random doubles from -1.0 to +1.0.

[Test]
public void MyTest(
    [Values(1, 2, 3)] int x,
    [Random(-1.0, 1.0, 5)] double d)
{
    ...
}

The following test fixture would be instantiated by NUnit three times, passing in each set of arguments to the appropriate constructor. Note that there are three different constructors, matching the data types provided as arguments.

[TestFixture("hello", "hello", "goodbye")]
[TestFixture("zip", "zip")]
[TestFixture(42, 42, 99)]
public class ParameterizedTestFixture
{
    private string eq1;
    private string eq2;
    private string neq;
    
    public ParameterizedTestFixture(string eq1, string eq2, string neq)
    {
        this.eq1 = eq1;
        this.eq2 = eq2;
        this.neq = neq;
    }

    public ParameterizedTestFixture(string eq1, string eq2)
        : this(eq1, eq2, null) { }

    public ParameterizedTestFixture(int eq1, int eq2, int neq)
    {
        this.eq1 = eq1.ToString();
        this.eq2 = eq2.ToString();
        this.neq = neq.ToString();
    }

    [Test]
    public void TestEquality()
    {
        Assert.AreEqual(eq1, eq2);
        if (eq1 != null && eq2 != null)
            Assert.AreEqual(eq1.GetHashCode(), eq2.GetHashCode());
    }

    [Test]
    public void TestInequality()
    {
        Assert.AreNotEqual(eq1, neq);
        if (eq1 != null && neq != null)
            Assert.AreNotEqual(eq1.GetHashCode(), neq.GetHashCode());
    }
}
Spelter answered 6/6, 2011 at 19:4 Comment(0)
H
39

Yes, there's a single parameter. You can make that parameter be arbitrarily complex, though. You could adapting the code from the documentation to use you Row type, for example:

class AndyTest : public ::testing::TestWithParam<Row> {
  // You can implement all the usual fixture class members here.
  // To access the test parameter, call GetParam() from class
  // TestWithParam<T>.
};

Then define your parameterized test:

TEST_P(AndyTest, CountViaDirectSQLCommand)
{
  // Call GetParam() here to get the Row values
  Row const& p = GetParam();
  std::string dbFilePath = testDBFullPath(p.dbname);
  {
    StAnsi fpath(dbFilePath);
    StGdbConnection db(p.fpath);
    db.Connect(p.fpath);
    int result = db.ExecuteSQLReturningScalar(StAnsi(p.command));
    EXPECT_EQ(p.numRecs, result);
  }
}

Finally, instantiate it:

INSTANTIATE_TEST_CASE_P(InstantiationName, AndyTest, ::testing::Values(
  Row("Empty.mdb", "select count(*) from collar", 0),
  Row("SomeCollars.mdb", "select count(*) from collar", 17),
  Row("SomeCollars.mdb", "select count(*) from collar where max_depth=100", 4)
));
Hermetic answered 6/6, 2011 at 19:11 Comment(5)
Thanks, Rob. Using a structure of some kind for the single parameter was what I feared would be the solution. As you can see from my edit to show the full test, it does make the gtest approach a lot bulkier than the mbUnit/NUnit style. I just realised there's also the commitment to ALL the tests for a given fixture having to be parameterized compared to it being a choice per test for the .Net ones. However, for people who are unwilling to use C++/CLI to test their native code, the gtest parameterization is still better than the nothing offered by other native kits!Spelter
Yes, but it's trivial to create another test fixture if you want another type of parameter. You can use inheritance if they all need similar setup and teardown code. If you don't have setup and teardown, then the fixture can be exactly the code I showed here for AndyTest. The bodies of the tests doesn't have to be much bulkier, as my edit shows by introducing the p parameter variable. You'll probably never get quite as elegant as .Net test frameworks since C++ doesn't support annotations, so you'll always have a few separate declarations.Hermetic
@RobKennedy Does this 'Row' attributes has documentation.. ? Can you point me to a link or something ?Distaff
@Vijay, it's a type introduced by this question's asker, not me. Besides, based on the usage demonstrated here, it's clear that the type doesn't really do anything. All it does is hold the values it's given. The asker hoped it could "magically" be split into multiple parameters when applied as a test attribute, but that obviously doesn't work. In my answer, I suggest that it should have fields for each of its construction arguments. You're welcome to write such a thing yourself; it's basic C++.Hermetic
@VijayC see hanselman.com/blog/MbUnitUnitTestingOnCrack.aspxSpelter
N
15

An alternative to using a custom structure as the parameter is to use the parameter generator ::testing::Combine(g1, g2, ..., gn). This generator allows you to combine the other parameter generators into a set of parameters with a type std::tuple that has a template type that matches the types of the values provided.

Note that this generator produces the Cartesian product of the values provided. That means that every possible ordered tuple will be created. I believe the original question is asking for a strict array of parameters with the provided values, which this does not support. If you need to have an array of strict parameters, you could use a tuple with the parameter generator ::testing::Values(v1, v2, ..., vN) where each value is a separate tuple.

Example:

#include <string>
#include <tuple>

class MyTestSuite : 
  public testing::TestWithParam<std::tuple<std::string, std::string, int>>
{

};

TEST_P(MyTestSuite, TestThatThing)
{
  functionUnderTest(std::get<0>(GetParam()), 
                    std::get<1>(GetParam()), 
                    std::get<2>(GetParam()));
  . . .
}

INSTANTIATE_TEST_SUITE_P(
  MyTestGroup,
  MyTestSuite,
  ::testing::Combine(
    ::testing::Values("FirstString1", "FirstString2"),
    ::testing::Values("SecondString1", "SecondString2"),
    ::testing::Range(10, 13)));

INSTANTIATE_TEST_SUITE_P(
  MyOtherTestGroupThatUsesStrictParameters,
  MyTestSuite,
  ::testing::Values(
    {"FirstString1", "SecondString1", 10},
    {"FirstString2", "SecondString2", 32},
    {"FirstString3", "SecondString3", 75}));

In the above example, the parameters created for MyTestGroup would look like the following:

[
  {"FirstString1", "SecondString1", 10},
  {"FirstString1", "SecondString1", 11},
  {"FirstString1", "SecondString1", 12},
  {"FirstString1", "SecondString2", 10},
  {"FirstString1", "SecondString2", 11},
  {"FirstString1", "SecondString2", 12},
  {"FirstString2", "SecondString1", 10},
  {"FirstString2", "SecondString1", 11},
  {"FirstString2", "SecondString1", 12},
  {"FirstString2", "SecondString2", 10},
  {"FirstString2", "SecondString2", 11},
  {"FirstString2", "SecondString2", 12}
]

Refer to the GoogleTest documentation for further details. (Accessed on 12/17/2019)

Nearby answered 17/12, 2019 at 15:36 Comment(2)
I change {"FirstString1", "SecondString1", 10} to make_tuple("FirstString1", "SecondString1", 10)Wholly
make_tuple() should be okay in most cases. Note that it will construct a tuple using implicit type deduction, then tries to convert that tuple to the test's template tuple type. Without make_tuple(), the initializer list will be used to create the tuple. Since the test template type is known, the tuple will be explicitly constructed. For most types, either should be fine. It's only a concern when the deduced type cannot be implicitly converted to the target type.Nearby

© 2022 - 2024 — McMap. All rights reserved.