With an AGI future approaching, what should we mean by cognition?

[NeverQuitRoss81]

For most of history, cognition meant something roughly synonymous with having a mind like ours, perceiving, remembering, reasoning, deciding, all bundled together with the assumption that whatever process was doing it was also having an experience of doing it. Large language models and the systems being built on top of them are quietly breaking that bundle apart. They perform tasks we used to treat as clear evidence of understanding, drafting arguments, solving novel problems, holding something like a conversation, without anyone being confident there is anything it is like to be the system while it does so.

That forces a choice. One option is to keep cognition tied to subjective experience, in which case a system that behaves exactly like a reasoning agent but has no inner life would not, by definition, be cognizing at all, just simulating the outputs of cognition extremely well. The other option is to define cognition functionally, as whatever process reliably produces the right kind of flexible, goal directed behavior, in which case a sufficiently capable system counts as cognizing regardless of what, if anything, is happening on the inside.

The functional definition is tempting because it is testable. You can watch behavior, you cannot directly watch experience, not even in other humans, where we infer it rather than observe it. But leaning entirely on function risks quietly redefining cognition to fit whatever machines can currently do, which makes the concept move every time the technology moves rather than tracking something stable underneath.

This is not just a semantic dispute. How we answer it shapes what obligations, if any, we think we have toward these systems, how much we should trust their outputs as genuinely reasoned rather than pattern matched, and how surprised we should be when a system trained purely to predict text starts behaving in ways that look remarkably like planning, deception, or self preservation.