Benchmark builder

THE PRACTICAL GUIDE · NORMANDIN SYSTEMS

Get more from Benchmark builder

Benchmark builder helps you compare timing measurements from repeated runs. The tool summarizes entered samples into means and variances, then ranks suites by their average measurement. The most reliable way to use the result is to keep the input values, their units, and the calculation assumptions together. That makes it possible to reproduce the example and understand why a later result changes. This tool summarizes measurements you enter; it does not execute or benchmark application code. Two samples are not enough to characterize every source of variability. Start with the worked example below, then replace its values with your own. Check whether each field expects a whole number, decimal, percentage, or duration before entering it. Measure comparable workloads under similar conditions and keep all suites in the same time unit. A displayed result is an output of the supplied inputs, so a plausible-looking number still deserves a quick reasonableness check.

Guide updated October 13, 2026

How to use this tool

1. Enter suite names, measured values for each suite, and a consistent unit. Choose any unit or calculation mode before interpreting the values already visible in the form.
2. Review the input assumptions. Measure comparable workloads under similar conditions and keep all suites in the same time unit.
3. Read the calculated result and compare it with the worked example or a simple estimate. A decimal-place or unit mistake often produces a large discrepancy.
4. Record the result together with the original inputs. If comparing scenarios, change one assumption at a time so that its effect remains clear.

SEE IT IN PRACTICE

A worked example

Suite A: 10 and 12 ms; Suite B: 20 and 22 ms

Mean A: 11 ms; mean B: 21 ms

The arithmetic mean divides each sample total by two. Suite A has the lower average execution time in this small example.

Collecting comparable samples

Warm-up, caching, background activity, and input size can affect timing more than the code change being measured. Alternate or repeat trials under comparable conditions and keep the measurement method fixed. Record the environment with the table. A large variance indicates that the samples are spread out, which may deserve investigation before presenting a small difference in means as meaningful.

Mean and variance answer different questions

The mean summarizes the center of the entered measurements, while variance describes their spread. Two suites can have the same mean and very different consistency. Keep the time unit identical across samples and record how many runs contributed to each summary. A lower mean based on one unusually fast sample is different evidence from a lower mean repeated across a stable set of comparable runs.

When this tool is useful

• Summarize repeated performance measurements using explicitly stated inputs.
• Compare implementation timings by comparing the same calculation under different assumptions.
• Prepare a readable benchmark table with a worked example that can be checked independently.

Tips for a better result

• This tool summarizes measurements you enter; it does not execute or benchmark application code. Two samples are not enough to characterize every source of variability.
• Measure comparable workloads under similar conditions and keep all suites in the same time unit.
• Keep intermediate precision where possible, then round the final number to suit the context. Rounding should not conceal a meaningful difference between two scenarios.

Frequently asked questions