How does a large language model write a sentence, one piece at a time?

By guessing the next piece over and over. A large language model is a neural network trained on a vast amount of text, and at heart it does one thing: given the text so far, it estimates how likely each possible next piece is. Is "I like to eat" more likely to be followed by "bread" or "rocks"? It picks a piece, adds it to the text, and asks the same question again. A whole answer is just that loop, run again and again.

The pieces are called tokens. Since the model only handles numbers, text is first split into tokens from a fixed vocabulary, often whole words or fragments of words, and each one becomes a number and then a list of numbers called an embedding.

The clever part is how the model uses context. Its transformer design has an attention mechanism that relates every token to every other one at once, however far apart they are, so the end of a long sentence can depend on its beginning. At the end, the model turns its calculation into a probability for every token in its vocabulary. A setting called temperature decides how adventurous the pick is: low means almost always the top choice, high means more randomness.

This explains a famous weakness. Training rewards a good guess at the next word even when the model lacks the information, so it can produce fluent, confident text that is false. When the data scientist Teresa Kubacka asked ChatGPT about a phenomenon she had made up, it answered with plausible-looking citations.