A controlled interpretability method for measuring how much a language model's text embeddings respond to writing style versus subject topic. Its key device is the bilingual QUENEAU-FENEON corpus, built by cross-generating from Queneau's Exercices de Style (one narrative in many styles) and Feneon's Nouvelles en trois lignes (many topics in one style) so that style and topic can be varied independently; embedding dispersion is then compared across models in French and English to reveal each model's differential sensitivity to stylistic information.
An interpretability study of how writing style, as opposed to subject topic, shapes the geometry of text embeddings produced by several state-of-the-art language models, compared across French and En…