Machine Learning Glossary: Plain Definitions of Common Terms

Updated · By Viviai

This machine learning glossary collects the terms that show up most often when beginners read about ML and AI, with short, plain definitions grouped by topic. Use it as a reference page: each entry states what the term means in one or two sentences, and linked words lead to full explainers. Definitions follow the usage in the Google Machine Learning Glossary, ISO/IEC 22989, and Stanford HAI.

Core concepts

Term Definition
Artificial intelligence (AI) The field of building systems that perform tasks normally associated with human intelligence, such as perception, reasoning, and language use.
Machine learning (ML) A branch of AI where models learn patterns from data instead of being given explicit rules.
Model A mathematical structure trained on data to make predictions or generate outputs.
Algorithm The procedure that learns a model's parameters from data, such as a decision tree or gradient descent.
Parameter A value the model learns on its own, such as a weight or bias in a neural network.
Hyperparameter A setting chosen before training, such as the learning rate or number of layers.
Deep learning Machine learning that uses neural networks with several layers.

Data and features

Term Definition
Dataset The collection of examples used for training, validation, and testing.
Training data The examples the model fits to during learning.
Validation data A held-out set used to tune settings and compare options.
Test data A separate set used only for a final estimate of performance.
Feature A measurable input property, such as age or pixel value.
Label The target answer the model learns to predict.
Instance A single data point or row, also called an observation.
Classification Predicting a category, such as spam or not spam.
Regression Predicting a continuous number, such as a price.
Supervised learning Learning from examples that include labels.
Unsupervised learning Finding structure in data without labels, such as clustering.

Training and optimization

Term Definition
Loss function A measure of how far a model's predictions are from the correct answers.
Gradient The derivative of the loss, indicating the direction that would increase it.
Gradient descent The iterative method that updates weights downhill to reduce loss.
Learning rate The multiplier that controls the size of each weight update.
Epoch One full pass over the entire training dataset.
Batch A subset of examples processed together before an update.
Backpropagation The algorithm that applies the chain rule to compute gradients across layers.
Epoch vs. iteration An epoch is a full dataset pass; an iteration is one single batch update.
Overfitting When a model fits the training data closely but does not generalize to new data.
Regularization Techniques that penalize complexity to reduce overfitting.
Gradient clipping Capping gradients at a maximum norm to prevent exploding updates.

Neural networks

Term Definition
Neural network A layered model of connected units loosely inspired by neurons.
Weight A parameter that scales an input as it passes between units.
Bias A parameter added to shift a unit's output up or down.
Activation function A nonlinear transform applied to a unit's output, such as ReLU.
Hidden layer An intermediate layer between input and output.
Batch normalization A technique that rescales layer activations during training.
Softmax A function that turns scores into probabilities that add to one.
Vanishing gradient When the backward signal shrinks so much that early layers stop learning.
Exploding gradient When the backward signal grows so large that training becomes unstable.

Generative AI and language

Term Definition
Generative AI Systems that produce new text, images, audio, or other content.
LLM (Large Language Model) A model trained on large text corpora to understand and generate language.
Token A unit of text the model reads, such as a word or part of a word.
Prompt The input instruction given to a generative model.
Hallucination Output that looks credible but is not grounded in fact or evidence.
Context window The maximum number of tokens a model can consider at once.
Fine-tuning Taking a pretrained model and training it further on a narrower dataset.
RAG (Retrieval-Augmented Generation) A method that fetches external documents before generating an answer.
Embedding A numeric vector that represents the meaning of text or other data.
AI agent A system that plans and takes actions toward a goal, often using tools.
Multimodal Working with more than one input type, such as text and images.

Evaluation

Term Definition
Accuracy The share of predictions that are correct.
Precision Among items predicted positive, the share that truly are.
Recall Among truly positive items, the share the model catches.
Confusion matrix A table showing correct and incorrect predictions across classes.
Generalization How well a model performs on data it has not seen before.
Cross-validation Splitting the data into rotating train and test sets for a steadier estimate.

How to use this glossary

Frequently asked questions

What is the difference between a parameter and a hyperparameter? The model learns parameters from data, such as weights. You choose hyperparameters in advance, such as the learning rate or the number of layers.

Are these definitions fixed across every source? The core meanings are stable, but wording differs between sources, and a few terms are used loosely in industry writing. When precision matters, check the linked standards and official documentation rather than relying on a single sentence.

Does this glossary cover every ML term? No. It focuses on the words beginners meet first. Narrow research terms and product-specific names are added as they become common enough to matter.

Key takeaways

Explore the full guides on machine learning, gradient descent, the learning rate, and neural networks.

Sources

  1. https://developers.google.cn/machine-learning/glossary
  2. https://hai.stanford.edu/ai-definitions
  3. https://www.ibm.com/docs/en/watsonx/saas?topic=glossary
  4. https://www.iso.org/obp/ui/es/?_escaped_fragment_=iso%3Astd%3A74296%3Aen

Learn by doing

Read a guide, then practice with bite-sized AI lessons you can take offline.

Start learning free