What is Language model, large language model?

A language model assigns probabilities to sequences of tokens, usually through the conditional probability of each token given the ones before it, $\P(w_t\mid w_1,\dots,w_{t-1})$; it generates text by sampling one token at a time. A large language model (LLM) is a transformer language model (chapter 8) with billions of parameters, pre-trained (chapter 10) on a large share of the public text of the internet and books, then fine-tuned to follow instructions.