Multiverse runs part of a 30B-parameter language model on an IBM quantum chip

Started by RelativityX, Today at 01:05 AM

Previous topic - Next topic

Rapid Andrew and 1 Guest are viewing this topic.

Topic: Multiverse runs part of a 30B-parameter language model on an IBM quantum chip   Views(Read 57 times)
Active members in this topic:
RelativityX(1) Rapid Andrew(1)

RelativityX

A quantum module has stood in for part of a large language model. Quantum Zeitgeist reports that Multiverse Computing and the University of Toronto replaced eight consecutive transformer blocks of Qwen3-30B-A3B-Instruct-2507, which has 48 layers, with a single quantum neural network module

The module has 145,281 trainable parameters and takes the place of 16.33 percent of the model's parameters. Perplexity on the WikiText-2 and C4 tests stayed within about 1 percent of the original model

The quantum parts ran on a 156 qubit IBM Heron, with 14 heads on 112 qubits, a two qubit depth of 13 and no routing swaps. The rest of the model ran on GPUs

Removing the quantum heads at a sensitive layer cost 3.5 perplexity points, but the classical parts alone came within 1 to 3 percent of the full module. The authors, including Román Orús and Hans-Arno Jacobsen, do not claim quantum advantage, and the work is an arXiv preprint

Swapping a sixth of a big model's parameters for a small module while keeping quality is interesting, whatever the quantum contribution turns out to be. The candid note that classical parts alone came close is welcome. It is an early experiment in combining language models with real quantum hardware

Could quantum modules ever make AI models smaller or cheaper? And how should we judge experiments that do not claim quantum advantage?

All original content unless stated in the loss log

Rapid Andrew

Running 14 heads on 112 qubits with no swaps shows careful circuit design. Hardware layout was respected. That keeps noise down

Still, perplexity gains came mostly from elsewhere. Interesting but early. A preprint needs review too