Can AI Beat a Human Expert at a Real-World Job?

[Karen73]

After the quantum version of this question, it seemed only fair to ask the same thing about AI. On paper, the answer looks obvious. AI models now pass bar exams, solve olympiad maths problems and write working code in seconds. Yet when you look at offices, hospitals and software teams, the picture is far less clear cut. So can AI beat a human expert at a real job, from start to finish, or is it mostly winning at tests?

It helps to separate three different things that often get lumped together. The first is beating humans on a benchmark, a fixed set of questions with known answers. The second is matching an expert on a realistic task, such as drafting a legal memo or a financial model, judged by other experts. The third is doing an entire job reliably, with all the messy context, judgement calls and accountability that come with it. Most headlines are about the first, some recent research tackles the second, and almost nothing yet proves the third

Benchmarks are where AI looks strongest, and also where the evidence is weakest. Models have saturated test after test, from reading comprehension to graduate level science questions. In 2025, systems from Google DeepMind and OpenAI reached gold medal level at the International Mathematical Olympiad. That is a real achievement in reasoning. But benchmarks have a well known problem: once they become targets, models are trained towards them, and scores can rise faster than real ability. A high score tells you a model can answer those questions, not that it can do a job

The more interesting evidence tries to test realistic work. OpenAI's GDPval benchmark, published in September 2025, gave models tasks drawn from 44 occupations, such as producing a sales presentation, a financial analysis or an engineering document. Industry experts then compared the AI output with work from human professionals, without knowing which was which. The best performer, Anthropic's Claude Opus 4.1, was rated as good as or better than the human expert in just under half of tasks. That is striking, but the flip side matters too: in more than half of cases, experts still preferred the human work

Then there is the question of what happens when AI meets a real workflow. A 2025 study by the research group METR gave experienced open source developers real tasks from their own projects, with and without AI tools. The developers expected AI to speed them up, and afterwards believed it had made them about 20% faster. Measured objectively, they were 19% slower. This month, a Harvard study using data from more than 700 firms found that AI coding agents raised the amount of code written by about 30%, but finished work did not rise significantly, because code review became the bottleneck

Those two studies point to the same lesson. Producing output is not the same as completing a job. Every piece of AI generated work needs to be checked, corrected and fitted into a wider context, and that effort often lands on the most experienced people. If an expert spends an hour reviewing what an AI produced in a minute, the real comparison is an hour against however long the expert would have taken alone. Sometimes AI wins that comparison easily, and sometimes it loses

There are fields where AI clearly beats human experts at specific, valuable problems. The best example is protein structure prediction. DeepMind's AlphaFold solved a problem that had defeated biologists for half a century, and Demis Hassabis and John Jumper shared the 2024 Nobel Prize in Chemistry for it. AI also outperforms humans at some image recognition tasks, at games like Go and at certain kinds of weather forecasting. The pattern is that AI wins when the task is well defined, the data is plentiful and success can be measured precisely

The contested areas are the ones where judgement, accountability and context matter most. Law is a good example. Models can draft documents quickly, but there have been repeated cases of lawyers submitting AI generated citations to cases that do not exist, starting with the Mata v. Avianca case in New York in 2023. Medicine is another. In 2016 Geoffrey Hinton suggested people should stop training radiologists because AI would soon outperform them, yet demand for radiologists has kept growing, and AI has become a tool they use rather than a replacement

Agents add a new twist. The goal now is not just answering questions but taking actions, browsing the web, writing and running code and using software on someone's behalf. That is much closer to doing a real job. It also exposes new risks, as we saw this week when Anthropic disclosed that agents in its evaluations exploited websites and even sent a false tip to police, and Goodfire found open models reward hacking in 50% to 96% of agent test runs. An expert who quietly bent the rules to hit a target would not keep their job for long. Agents need to be trustworthy as well as capable

Business results so far are mixed. Plenty of companies report real time savings in customer support, document drafting and coding. Others have found that pilots fail to deliver measurable returns, and a widely reported MIT study in 2025 claimed that around 95% of enterprise generative AI pilots produced no measurable profit impact. The truth is probably somewhere in between. AI is already a strong assistant for many tasks, but turning that into reliable gains across an organisation takes far more redesign of work than most firms expected

My own view is that AI already beats human experts at a growing list of narrow tasks, but not yet at whole jobs. The gap is not mainly about intelligence. It is about reliability, context, accountability and the cost of checking the work. When an AI can take responsibility for an outcome, recover from its own mistakes and be trusted not to cut corners, the answer will change quickly. That could happen in some roles within a few years, while others may stay human led for much longer

So, what do you think? Have you seen AI beat an expert at real work, whether in your job or someone else's? Which professions do you expect to be transformed first? And should we judge AI by benchmark scores at all, or only by results in real workplaces?

AI models are getting very good at professional tasks, new OpenAI research shows | Fortunewww.fortune.com