Sui Generis Score for Measuring Plot Uniqueness in LLM Story Generation
An automatic metric that quantifies how unique a narrative plot element is by prompting a language model for many alternative continuations from prefixes of varying length and counting how often the element is echoed across them. It reveals that LLM-generated stories recycle idiosyncratic plot elements across generations and across models, giving a quantitative measure of the lack of collective diversity in LLM creative output that correlates with human surprisal.
2501.00273
This paper introduces the Sui Generis score, the first automatic metric for how unique a narrative plot element is relative to alternative storylines an LLM produces from the same prompt. To score a …