A new CLOPSh benchmark measures how fast quantum chips actually sustain calculations, and the numbers are humbling

[ThreadNecro11]

Most quantum computing benchmarks to date have focused heavily on fidelity, essentially how accurately a quantum processor executes a given operation, while largely ignoring a separate and arguably more practically important question, how quickly these machines can actually run complex calculations over and over again in a sustained way. A new benchmark called CLOPSh, short for Circuit Layer Operations Per Second, developed by Andrew Wack alongside Riverlane Computing, sets out specifically to close that gap.

The benchmark works by tracking how quickly a quantum system completes physical layers, meaning parallel slices of qubit disjoint two qubit gates that are separated by synchronisation barriers as the circuit executes. That system level framing matters a great deal for the kind of workloads increasingly central to practical quantum computing, particularly layered, parameterised circuits that get executed repeatedly within classical quantum hybrid workflows such as variational algorithms and error mitigated simulations, exactly the categories of application most quantum computing companies are currently betting their near term commercial relevance on.

Testing a superconducting processor against the new benchmark produced a sustained rate of 74,863 circuit layers per second, which the researchers describe as a threefold improvement over what prior execution rate measurement methods had shown. Layer fidelity benchmarks run alongside the throughput measurement averaged 98.7 percent, giving a reasonably complete picture of both how fast and how accurately the processor was actually performing under sustained, repeated operation rather than in a single isolated run.

The honest caveat buried in the research matters just as much as the headline throughput number though. The paper is explicit that current figures represent benchmarked capability on relatively simple layered circuits and do not yet reflect execution speeds achievable once you factor in full classical control overhead or the complex, application specific compilation routines that any genuinely challenging real world problem would actually require. In other words, this is a clean laboratory measurement of raw sustained throughput under simplified conditions, not a preview of what performance looks like once a quantum computer gets pointed at an actual messy scientific or industrial problem.

That distinction is exactly why this kind of benchmark matters for the field's credibility going forward. As quantum computing moves further from purely academic demonstrations toward genuine industrial deployment, having a rigorous, standardised way to measure sustained real world throughput rather than just peak fidelity on cherry picked test circuits becomes essential for anyone trying to compare competing systems honestly, or for customers trying to work out whether a given quantum computer can actually deliver useful results at the speed and repetition their application demands.

Benchmark Quantum Reveals Limitationsphys.org

[Brittle Ruby]

Variational algorithms and error mitigated simulations being the specific workloads this benchmark targets makes sense given how much near term commercial hope in this industry is currently riding on exactly those two categories of application rather than full fault tolerant quantum computing which is still years away.