feat: add multi-spec execution with result aggregation and timeseries replay#229
Merged
fuweid merged 2 commits intoAzure:mainfrom Jan 23, 2026
Merged
Conversation
… replay example Add support for executing multiple specs sequentially within a single kperf runner invocation, enabling time-series replay scenarios. Changes: - Add MultiSpecRunnerMetricReport type for multi-spec results - Implement executeSpecs() for sequential spec execution - Implement aggregateResults() to combine per-spec results - Add printMultiSpecResults() for multi-spec output format - Add hasCliOverrides() to detect and prevent CLI overrides with multi-spec - Update buildRunnerGroupSummary() to handle MultiSpecRunnerMetricReport format - Add timeseries_replay example with 3 phases: baseline (10 QPS, 30s), spike (100 QPS, 20s), recovery (25 QPS, 30s) - Add benchTimeseriesReplayCase command to demonstrate time-series replay This enables realistic workload patterns that vary over time, useful for stress testing, capacity planning, and validating system behavior under changing load conditions. Signed-off-by: JasonXuDeveloper <jason@xgamedev.net>
fuweid
approved these changes
Jan 23, 2026
Signed-off-by: JasonXuDeveloper <jason@xgamedev.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add support for executing multiple specs sequentially within a single kperf runner invocation, enabling time-series replay scenarios where workload patterns vary over time.
Changes
Core Multi-Spec Functionality
MultiSpecRunnerMetricReporttype for multi-spec results inapi/types/metric.goexecuteSpecs()for sequential spec executionaggregateResults()to combine per-spec results into aggregated metricsprintMultiSpecResults()for multi-spec output formathasCliOverrides()to detect and prevent CLI overrides with multi-spec configsbuildRunnerGroupSummary()inrunner/utils.goto handle both single and multi-spec report formatsExample and Demo
benchTimeseriesReplayCasecommand to demonstrate time-series replayTechnical Details
Use Cases
This enables realistic workload patterns useful for:
Testing
Related
This PR builds on #226 which converted the
Specfield toSpecsarray format.