Emergent introspective awareness in large language models \ Anthropic

Q: Couldn’t the models just be making up answers to introspective questions? This is exactly the question we designed our experiments to address. Models are trained on data that includes examples of people introspecting, so they can certainly act introspective without actually being introspective. Our concept injection experiments distinguish between these possibilities by establishing known ground-truth information about the model’s internal states, which we can compare against its self-reported states. Our results suggest that in some examples, the model really is accurately basing its answers on its actual internal states, not just confabulating. However, this doesn’t mean that models always accurately report their internal states—in many cases, they are making things up!