How to special-case the number of iterations in google benchmark?

Question

I am aware of the --benchmark_repetitions flag and it is not what I need. I want to be able to specify the number of iterations for one benchmark. I am okay with using a --benchmark_iterations flag which sets the number of iterations for all benchmarks.

I know google benchmark is smart to figure out how many iterations is needed to get a good measurement. This is good enough for most use cases but my use case is different. I need to be able to control the number of iterations precisely.

Welborn

Answer 1

BENCHMARK(YourBenchmark)->Iterations(num_ite); will do the trick.

You can specify iterations for each benchmark.

If you are using BENCHMARK_F, do that in the constructor

class BenchmarkBase : public benchmark::Fixture {
public:

    BenchmarkBase() {
        Iterations(num_ite);
    }
};

Vesperal

Or benchmark::RegisterBenchmark(...)->Iterations(num_ite) for custom main()'s. – Unguentum

I tried adding Iterations(num_ite) in either constructor or SetUp() but nether worked. – Welborn

FWIW -- adding it to SetUp() did not work, but adding it to the constructor did work for me. – Hanger

Answer 2

It doesn't support the case of wanting to tune the iteration count precisely.

As you've pointed out, it runs until there are enough iterations to get a good enough signal (though it doesn't yet actually check the statistics). We used to have a way to set the iteration counts, but the interaction with the timing related flags are complex and were confusing to people trying to configure their benchmarks, so they were removed.

It is something that comes up often though, and if there is a way we could support iterations and timing together, or support iterations but also warn if we think the result isn't meaningful, then i'm open to considering it.

Dirk

Answer 3

Since version 1.8.0, Google Benchmark now support specifying the number of iteration directly from the command line. This is done through the --benchmark_min_time option. Simply specify the number of iteration followed by x.

For example, to run all benchmark with only one iteration each:

--benchmark_min_time=1x

The change was made here: https://github.com/google/benchmark/pull/1525

Exotoxin