Why an AI that can pass the bar exam still cannot reliably count the letters in a simple word

[Morpheus49]

Ask most AI chatbots how many times the letter R appears in strawberry and there is a good chance they will confidently answer two, when the correct answer is three. This became a famous, slightly embarrassing internet joke, a machine capable of writing working code and passing professional licensing exams stumbling over something a child could get right. The actual explanation has nothing to do with the AI being unintelligent, and everything to do with how it actually reads text in the first place

Language models do not see individual letters the way you do. Before any text reaches the model, it gets chopped up into chunks called tokens, sometimes a whole word, sometimes just a fragment of one. Strawberry might become a single token, or split into something like straw and berry, but either way, the model never receives the individual letters s-t-r-a-w-b-e-r-r-y, it receives a couple of opaque numerical IDs standing in for those chunks. Asking it to count the Rs inside a token is a bit like asking someone to count the atoms inside a single Lego brick, they can see the brick clearly, but the atoms simply are not visible at that level

This tokenization quirk explains a surprising number of other odd AI behaviors too, why some models occasionally stumble on basic arithmetic with large numbers, why certain rare or unusual words get misspelled, and why costs for using these models are measured in tokens rather than words or characters. Newer models that pause to reason step by step before answering have gotten noticeably better at this specific test, essentially working around the limitation by recalling the spelling from memory and counting deliberately rather than trying to see letters that were never actually there to begin with

Why Most AI Models Can't Count the Letter "R" in Strawberryjoshmcdonald.medium.com