-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Description
Didn't see a discussion tab, so I'm asking here:
Catch2 included some rudimentary benchmarking, but didn't include patterns for checking benchmarking results in order to detect performance regresses - have you thought about something along these lines, or do you think this is out of your scope here?
It will probably need benchmarking data to be recorded, merged, and carried over between test runs.
Here's a cobbled up example from the top of my head to give more context:
auto result = BENCHMARK_ADVANCED("default construction") (timer meter) {
some<uninitialized<T>> storage(meter.runs());
meter.measure([&](int i) {
return storage[i].construct();
});
};
REQUIRE(result.within(0.1f));
REQUIRE(result.under(5s));
REQUIRE(result.faster_than(result2));
//etc...I'm not familiar how other projects do their benchmarking and performance regress tests, but I suppose this is one way to do it?
I might be able to work on it, if you decide this is a feature you want. With C++23's chrono improvements, I believe it can be made quite painless.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels