The thesis, argued in philosophy of cognitive science, that large language models do not merely memorize training patterns or act as a stochastic look-up table but partly process information via internal representations - states that carry mutual information about task-relevant variables and are causally exploited to guide behavior. It supplies an information-theoretic criterion distinguishing genuine representation from finite-state or look-up-table memorization, marshals interpretability evidence (for example probing classifiers that recover latent board state in Othello-GPT), and defends a methodology of training probes on internal activations and causally intervening on them to identify representations and build explanations. The position grounds higher-level claims about whether such models have beliefs, concepts, knowledge, and understanding.
This philosophy paper (forthcoming in Ergo) asks what kind of algorithm large language models implement: whether their behavior is driven partly by representation-based information processing of the …