The AI model matters less than the harness wrapped around it, according to a growing chorus of CIOs

[Jonathan]

A recent piece in the Wall Street Journal's CIO Journal lays out a shift happening quietly inside enterprise AI teams, the model powering a given AI product increasingly matters less than the software layer wrapped around it, commonly called the harness. That harness includes things like system prompts, tool definitions, memory, permission policies, and the orchestration logic that actually lets an AI agent do useful work rather than just answer a single question.

Supporting research from AI consultancy Systima found something that should worry any CIO comparing vendors purely by model benchmark scores. Running the exact same underlying model, Claude Sonnet 4.5 in this case, through two different harnesses, Anthropic's own Claude Code versus the open source OpenCode, produced sharply different token overhead purely because of differences in how each harness was configured. Same model, wildly different cost and behavior, entirely because of the scaffolding around it.

This reframes a lot of what enterprises should actually be evaluating when they pick an AI vendor. Comparing raw model capability between providers increasingly misses the point if the harness quality varies just as much or more between competing products built on the exact same underlying model. A well designed harness can make a mediocre model perform like a great one on a specific task, and a poorly designed harness can waste enormous amounts of money and effort even wrapped around the best model available.

The competitive implications go further than efficiency too. Companies with harnesses deeply wired into a customer's own data, permissions, and workflows create a kind of lock in that has nothing to do with which underlying model is technically the smartest. That's arguably why Anthropic's enterprise revenue keeps climbing even in periods where its raw benchmark scores aren't the industry's best, the thing customers actually depend on day to day is the surrounding system, not the isolated model score sitting at the center of a leaderboard

Introducing The Ai Model Harness A638C65Bwww.wsj.com

[Laura94]

Systima's finding about identical models producing wildly different token overhead through different harnesses should be required reading for any procurement team still picking AI vendors off a benchmark leaderboard. The model score tells you almost nothing about what you'll actually pay to run it

[Luca73]

This lines up with something I've noticed switching between different coding assistants that technically run the same underlying model. The experience varies enormously depending on how the tool manages context, memory, and tool calling around that model. It's basically confirmation that the wrapper matters as much if not more than the engine inside it

[Marcus]

Curious whether this dynamic ends up commoditizing the underlying models even faster than expected, since if the harness is where the real value and differentiation lives, model providers might end up racing each other down to cost purely on compute efficiency. That would be a pretty different competitive landscape than the current one, which still gets framed mostly around whose model benchmarks best. Feels like a meaningfully different endgame than most people are currently planning around

[BiasField78]

The zero data retention point buried in some of the broader coverage around this topic deserves more attention than it usually gets. If the harness is where enterprise value actually concentrates, then trust around what happens to a company's proprietary data flowing through that harness becomes the single most important thing CIOs should be pushing vendors on.

Every enterprise software vendor spent the last twenty years building trust that customer data stays customer data, and AI harnesses are forcing a version of that same conversation all over again from scratch. Whoever gets that trust question locked down cleanly first probably wins a disproportionate share of enterprise contracts regardless of whose underlying model is marginally smarter. Worth watching which vendors actually commit to hard guarantees here versus who just talks about it in marketing copy

[NeutronDrift]

I think this explains a pattern that's confused a lot of people watching the AI industry from outside, why does Anthropic keep growing enterprise revenue even in stretches where a competitor's model technically scores higher on some benchmark. The answer according to this framing is that enterprises aren't actually buying the model in isolation, they're buying the whole system around it.

Once a company builds its workflows, permissions, and data pipelines around a specific harness, switching to a technically superior model elsewhere means rebuilding all of that infrastructure from scratch. That's a much bigger switching cost than most outside observers seem to appreciate when they frame this purely as a model quality race

[Slow Hollow]

Wonder how much of this framing is just consultancies and vendors trying to make their own harness product sound more essential than it actually is. Not saying the underlying point is wrong, just that everyone quoted here has some financial stake in convincing CIOs the harness matters as much as they're claiming

[Jan1]

Lock in through harness design rather than model quality is a much more durable moat for these companies long term. Models get commoditized fast, but a harness deeply wired into someone's actual data and workflows is genuinely hard to rip out