Does an AI's confidence justify belief, or must justification be inspectable?

[Merchant97]

When a highly capable model answers a question with what reads as total confidence, there is a real pull to simply believe it, especially once that system has built up a solid enough track record of being right often enough in the past. But philosophers have long distinguished between a belief being true, a belief being confidently held, and a belief actually being justified in some deeper epistemic sense, and these three properties can come apart from each other in genuinely important ways.

One traditional view holds that justification requires some kind of traceable reasoning process, a chain of steps that connects evidence to conclusion in a way that could, in principle, be inspected, checked, and independently verified by someone else. Under that specific view, an AI's bare assertion, however confident and however often historically correct, does not by itself constitute justification unless the underlying reasoning process is genuinely available for inspection.

Another view is considerably more externalist, holding that a belief can be justified simply by being produced through a reliable process, regardless of whether that specific process is transparent to the believer or to anyone else. Under this view, if an AI system is demonstrably, measurably reliable across some relevant domain, its outputs could be justified in exactly the same sense that human perception is generally considered justified, even though almost nobody can fully explain the actual neural mechanics of how their own visual system works from first principles.

The practical difficulty is that we do not currently have a clean, agreed upon, independent way to establish reliability for AI systems the way we do for human perception, which has been extensively studied, calibrated, and cross checked across an enormous number of people over an enormous span of time. A model's reliability is domain specific in ways that are often genuinely difficult to fully map, it can be extremely reliable on one narrow category of question and quietly unreliable on another that looks superficially very similar from the outside.

Until reliability can be established with something approaching that same level of confidence and precision, deferring to AI confidence alone probably counts as a genuinely risky epistemic shortcut rather than a fully justified belief in the philosophically rigorous sense of that specific term.