Memorisation is a model’s ability to reproduce specific training examples, including their outcomes, rather than a rule that generalises; language models memorise text seen even once (Carlini and co-authors, 2021). The anonymisation test replaces the identifiers in an input (company names, tickers, dates) by placeholders and measures how much the model’s accuracy falls: a model that reads the text loses little, a model that recalls the event loses its memory.
ml_llm.contamination.