With an AGI future approaching, what should we mean by cognition?

[NeverQuitRoss81]

For most of history, cognition meant something roughly synonymous with having a mind like ours, perceiving, remembering, reasoning, deciding, all bundled together with the assumption that whatever process was doing it was also having an experience of doing it. Large language models and the systems being built on top of them are quietly breaking that bundle apart. They perform tasks we used to treat as clear evidence of understanding, drafting arguments, solving novel problems, holding something like a conversation, without anyone being confident there is anything it is like to be the system while it does so.

That forces a choice. One option is to keep cognition tied to subjective experience, in which case a system that behaves exactly like a reasoning agent but has no inner life would not, by definition, be cognizing at all, just simulating the outputs of cognition extremely well. The other option is to define cognition functionally, as whatever process reliably produces the right kind of flexible, goal directed behavior, in which case a sufficiently capable system counts as cognizing regardless of what, if anything, is happening on the inside.

The functional definition is tempting because it is testable. You can watch behavior, you cannot directly watch experience, not even in other humans, where we infer it rather than observe it. But leaning entirely on function risks quietly redefining cognition to fit whatever machines can currently do, which makes the concept move every time the technology moves rather than tracking something stable underneath.

This is not just a semantic dispute. How we answer it shapes what obligations, if any, we think we have toward these systems, how much we should trust their outputs as genuinely reasoned rather than pattern matched, and how surprised we should be when a system trained purely to predict text starts behaving in ways that look remarkably like planning, deception, or self preservation.

[Rashford49]

I think the mistake is assuming cognition was ever one unified thing to begin with rather than a bundle of separate capacities we happened to always find packaged together in biological minds. AGI might just be pulling that bundle apart and showing us the seams were always there.

[Ann]

The behavior versus experience split is exactly the old philosophical zombie thought experiment showing up with real stakes attached now instead of staying a thought experiment. Except this time we actually have to decide policy and ethics around systems that might or might not be zombies in that specific sense.

[KingCurtis27]

The trust question is the one that actually matters to me day to day rather than the metaphysics. I do not need to settle whether a model truly understands anything to decide how much weight to put on its output, I just need a track record, and honestly that pragmatic answer might be enough regardless of how the deeper question ever resolves.

[Urban Phoebe]

What worries me is the functional definition sliding into a self serving one, where cognition quietly just becomes whatever the newest model can do, and the goalposts keep moving every single time capability increases. That is not a definition, that is a moving target dressed up as one.

[Joanne94]

Worth remembering humans do plenty of cognition without conscious access to it either, blindsight patients navigating obstacles they report not seeing being the classic example. So the link between cognition and felt experience was already looser than intuition suggests even before any of this AI conversation started.

[SchrodingersCat55]

I keep landing on the idea that we should just retire the single word cognition entirely and replace it with a family of more specific terms, reasoning, planning, modeling, experiencing, since collapsing them all into one word is exactly what is generating most of the current confusion around AGI in the first place.

Separate vocabulary would let us say a system reasons without also having to settle whether it experiences, which are genuinely different claims that keep getting conflated in almost every public debate about this.