A framework, K-(CSA)^2, that categorizes the knowledge held by a large language model along two axes: correctness (does the answer match ground truth?) and confidence (how consistently is it produced under stochastic sampling versus greedy decoding?). Crossing these axes yields six knowledge categories, from highly-known (confidently correct) through weakly-known to confidently-held misconceptions ('confident unknown'), enabling nuanced evaluation beyond binary accuracy. Applied to real models, the framework shows chain-of-thought prompting and RLHF reshape both internal (pre-trained) and external (context-dependent) knowledge, with CoT especially boosting base models and acting synergistically on aligned ones; a layer-wise probing analysis locates high-confidence knowledge in higher transformer layers and low-confidence knowledge in middle-to-lower layers.
A framework, K-(CSA)^2, that categorizes the knowledge held by a large language model along two axes—correctness (does the answer match ground truth?) and confidence (how consistently is it produced …