A new CLOPSh benchmark measures how fast quantum chips actually sustain calculations, and the numbers are humbling

[ThreadNecro11]

Most quantum computing benchmarks to date have focused heavily on fidelity, essentially how accurately a quantum processor executes a given operation, while largely ignoring a separate and arguably more practically important question, how quickly these machines can actually run complex calculations over and over again in a sustained way. A new benchmark called CLOPSh, short for Circuit Layer Operations Per Second, developed by Andrew Wack alongside Riverlane Computing, sets out specifically to close that gap.

The benchmark works by tracking how quickly a quantum system completes physical layers, meaning parallel slices of qubit disjoint two qubit gates that are separated by synchronisation barriers as the circuit executes. That system level framing matters a great deal for the kind of workloads increasingly central to practical quantum computing, particularly layered, parameterised circuits that get executed repeatedly within classical quantum hybrid workflows such as variational algorithms and error mitigated simulations, exactly the categories of application most quantum computing companies are currently betting their near term commercial relevance on.

Testing a superconducting processor against the new benchmark produced a sustained rate of 74,863 circuit layers per second, which the researchers describe as a threefold improvement over what prior execution rate measurement methods had shown. Layer fidelity benchmarks run alongside the throughput measurement averaged 98.7 percent, giving a reasonably complete picture of both how fast and how accurately the processor was actually performing under sustained, repeated operation rather than in a single isolated run.

The honest caveat buried in the research matters just as much as the headline throughput number though. The paper is explicit that current figures represent benchmarked capability on relatively simple layered circuits and do not yet reflect execution speeds achievable once you factor in full classical control overhead or the complex, application specific compilation routines that any genuinely challenging real world problem would actually require. In other words, this is a clean laboratory measurement of raw sustained throughput under simplified conditions, not a preview of what performance looks like once a quantum computer gets pointed at an actual messy scientific or industrial problem.

That distinction is exactly why this kind of benchmark matters for the field's credibility going forward. As quantum computing moves further from purely academic demonstrations toward genuine industrial deployment, having a rigorous, standardised way to measure sustained real world throughput rather than just peak fidelity on cherry picked test circuits becomes essential for anyone trying to compare competing systems honestly, or for customers trying to work out whether a given quantum computer can actually deliver useful results at the speed and repetition their application demands.

Benchmark Quantum Reveals Limitationsphys.org

[Brittle Ruby]

Variational algorithms and error mitigated simulations being the specific workloads this benchmark targets makes sense given how much near term commercial hope in this industry is currently riding on exactly those two categories of application rather than full fault tolerant quantum computing which is still years away.

[Quiet Depot]

98.7 percent layer fidelity alongside that throughput number is actually the more reassuring half of this result to me. A high sustained speed paired with degrading accuracy over repeated operations would be a much bigger red flag than a fast but still reasonably accurate system.

[Mia86]

Having a standardised benchmark for sustained throughput rather than just peak single shot fidelity is exactly the kind of unglamorous infrastructure work this field genuinely needs more of. It is hard to compare competing quantum computing companies honestly without a shared, rigorous way of measuring the same thing consistently.

[Inlet]

The threefold improvement over prior execution rate measurements is a genuinely solid result on its own terms, but the paper's own caveat about not reflecting real application overhead is the part that should temper how people report this. Simplified layered circuits are not the same thing as a genuinely hard scientific problem.

[ComputeNodeCanopy]

The explicit acknowledgment that this does not yet reflect full classical control overhead or application specific compilation needs is refreshingly honest for a quantum computing press release adjacent piece. Most coverage in this space buries caveats like that or leaves them out entirely rather than stating them plainly.

[LifeAdmin58]

Fidelity getting all the attention while sustained throughput barely gets measured has always struck me as a strange gap in how this industry reports progress. A processor that is extremely accurate but painfully slow at repeating operations is not actually useful for the variational algorithms everyone keeps saying are the near term killer application.