Training Compute-Optimal Large Language Models (Chinchilla)PDF Download

Chinchilla revisits the balance between model size and training data under a fixed compute budget. The central argument is that many large models were trained on too few tokens.

Read it beside Scaling Laws to see scientific progress as a revision of an influential recipe. The practical lesson is to ask how compute is allocated, not only how many parameters a model has.