How does ChatGPT actually decide what word to say next?

[Scholar29]

At the core of every response is a genuinely simple sounding step, the model looks at everything typed so far and calculates a probability score for every possible next word or word fragment in its vocabulary. It then picks one of the higher probability options, not always the single highest one, and repeats that exact same process for the following word, building the whole response one small step at a time.

That probability calculation comes from a neural network trained on an enormous amount of text, where the model adjusted billions of internal parameters over training specifically to get better at predicting what word plausibly comes next in real human writing. Nothing about that training process involves looking anything up in a database, the model has no live access to facts, it is purely working from patterns baked into those parameters during training.

A setting called temperature controls how much randomness gets introduced into that word selection. A low temperature setting picks the highest probability word almost every time, producing safer and more predictable output, while a higher temperature setting allows genuinely lower probability words to occasionally get picked, producing more varied and sometimes more creative but also less predictable results.

This word by word process is exactly why these models can produce a confident sounding sentence that turns out to be factually wrong, since the model is optimizing for what word is statistically plausible next, not for what is actually true. It has no separate internal fact checking step running alongside the word prediction itself.

So each response you see is really the end result of thousands of individual small probability calculations chained together one after another. Not a single planned out answer the model composed all at once before starting to type it

[StormForge62]

TLDR, it predicts one word at a time based on probability. Picks one, then repeats using everything written so far including its own new word, there is no separate fact checking step running alongside that process

[Solo Lantern]

The temperature setting explanation finally made sense of something I had noticed but never understood.

Same exact prompt sometimes gives a noticeably different answer, and that randomness setting is apparently exactly why