Logits
Raw unnormalized scores from the final linear layer of a model, converted to probabilities via softmax.
type: glossary title: "Logits" tags: ["glossary", "output", "classification"] created: 2025-01-01
Logits
Definition: The raw, unnormalized scores output by the final linear projection layer of a model before any softmax normalization is applied; in a language model, logits form a vector of length equal to the vocabulary size, where each value represents the model's unnormalized preference for the corresponding token.
Used in: Transformer, Self-Attention
Do not confuse with: Probabilities (which are logits after softmax normalization) or log-probabilities (the logarithm of probabilities, sometimes loosely called logits in older literature).