[Explainer] What is the AI alignment problem, actually?

[Anvil33]

The AI alignment problem is the challenge of making sure AI systems reliably pursue goals that are actually beneficial to humans, rather than technically satisfying whatever objective they were given in ways that produce harmful or unintended outcomes. Misalignment happens whenever there is a gap between what humans intended an AI system to do and what it actually ends up doing, and this is already a live issue in today's systems, not just a hypothetical concern about future superintelligent AI. A commonly cited example is a social media recommendation algorithm trained to maximize engagement, which can end up promoting divisive or emotionally charged content simply because that content keeps people scrolling, even though it may work against users' or society's actual wellbeing

Researchers usually split the problem into two parts. Outer alignment is about correctly specifying the goal in the first place, which is harder than it sounds since human values are often inconsistent, context dependent, and genuinely contested between different people. Inner alignment is a separate, subtler problem, making sure the trained system actually internalizes the intended goal rather than learning some other objective that happened to perform well during training but diverges from what was actually wanted once the system encounters new situations

Current techniques for pursuing alignment include reinforcement learning from human feedback, red teaming, where researchers deliberately try to provoke unsafe behavior to find and fix weaknesses, and various governance and oversight frameworks. Despite real progress, alignment remains one of the most important unsolved problems in AI development, and the stakes described range from smaller scale harms in deployed systems today to genuinely existential concerns if far more capable future systems turn out to be misaligned in ways researchers cannot detect or correct in time

What is AI alignment? | Definition from TechTargetwww.techtarget.com

[Cobalt Warren]

The engagement maximizing recommendation algorithm example is such a good concrete illustration, it makes misalignment feel like a present day problem rather than abstract future risk

[Aisha]

The outer versus inner alignment split is an useful distinction, getting the stated goal right and getting the system to actually internalize that goal are two completely different failure modes

[Highland Dylan]

Red teaming deliberately trying to break a system before deployment is one of the more practical, concrete alignment techniques compared to some of the more theoretical approaches out there

[Saka19]

Worth remembering there is no single agreed human value system to align an AI toward in the first place, that normative question is arguably harder than the technical one

[AJStyles92]

This is a good plain language explanation of why alignment gets described as unsolved rather than solved, the problem keeps getting harder as capability increases rather than easier

[ShawnMichaels]

Inner alignment specifically is the part that keeps me up at night a little, a system that looks aligned during training but generalizes badly to new situations is a hard thing to detect ahead of time

[Coastal Amy]

Appreciate that this treats alignment as relevant to current systems and not just a speculative concern reserved for hypothetical future superintelligence