Artificial intelligence is rewriting the world, and simultaneously inventing a whole new language to describe how it is doing it. Sit in on any product meeting, pitch, or panel these days, and you will hear people toss around LLMs, RAG, RLHF, and a dozen other terms that can make even very smart people in the tech world feel a little insecure. This AI glossary is our attempt to fix that: plain-English definitions of the terms you are most likely to actually run into, whether you are building with this stuff, investing in it, or just trying to keep up by reading TechCrunch or listening to related podcasts. We update it regularly as the field evolves, so consider it a living document, much like the AI systems it describes.<\/p>\n
Artificial general intelligence, or AGI, is a nebulous term. It generally refers to AI that is more capable than the average human at many, if not most, tasks. OpenAI<\/a> CEO Sam Altman once described AGI as the “equivalent of a median human that you could hire as a co-worker.” Meanwhile, OpenAI’s charter defines AGI as “highly autonomous systems that outperform humans at most economically valuable work.” Google DeepMind’s understanding differs slightly; the lab views AGI as “AI that’s at least as capable as humans at most cognitive tasks.” Confused? Not to worry \u2014 even the experts at the forefront of AI research have no definitive answer.<\/p>\n A neural network refers to the multi-layered algorithmic structure that underpins deep learning and, more broadly, the whole boom in generative AI tools following the emergence of large language models. Although the idea of taking inspiration from the densely interconnected pathways of the human brain as a design structure for data processing algorithms dates back to the 1940s, it was the much more recent rise of graphical processing hardware (GPUs) \u2014 via the video game industry \u2014 that really unlocked the power of this theory. These chips proved well suited to training algorithms with many more layers than was possible in earlier epochs, enabling neural network-based AI systems to achieve far better performance across many domains, including voice recognition, autonomous navigation, and drug discovery.<\/p>\n Deep learning is a subset of self-improving machine learning in which AI algorithms are designed with a multi-layered, artificial neural network (ANN) structure. This allows them to make more complex correlations compared to simpler machine learning-based systems, such as linear models or decision trees. Deep learning AI models are able to identify important characteristics in data themselves, rather than requiring human engineers to define these features. The structure also supports algorithms that can learn from errors and, through a process of repetition and adjustment, improve their own outputs. However, deep learning systems require a lot of data points to yield good results (millions or more). They also typically take longer to train compared to simpler machine learning algorithms, so development costs tend to be higher.<\/p>\n Although somewhat of a multivalent term, compute generally refers to the vital computational power that allows AI models to operate. This type of processing fuels the AI industry, giving it the ability to train and deploy its powerful models. The term is often a shorthand for the kinds of hardware that provides the computational power \u2014 things like GPUs, CPUs, TPUs, and other forms of infrastructure that form the bedrock of the modern AI industry.<\/p>\n Large language models, or LLMs, are the AI models used by popular AI assistants, such as ChatGPT, Claude, Google’s Gemini<\/a>, Meta’s AI Llama, Microsoft Copilot, or Mistral’s Le Chat. When you chat with an AI assistant, you interact with a large language model that processes your request directly or with the help of different available tools, such as web browsing or code interpreters. LLMs are deep neural networks made of billions of numerical parameters (or weights) that learn the relationships between words and phrases and create a representation of language, a sort of multidimensional map of words. These models are created from encoding the patterns they find in billions of books, articles, and transcripts. When you prompt an LLM, the model generates the most likely pattern that fits the prompt.<\/p>\n Mixture of Experts is a model architecture that splits a neural network into many smaller specialized sub-networks, or “experts,” and only activates a handful of them for any given task. Rather than routing every request through the entire model \u2014 like calling in your whole office for every question \u2014 an MoE model has a built-in “router” that picks just the right specialists for the job. This makes it possible to build enormous models that stay relatively fast and cheap to run, since only a fraction of the network is doing work at any one time. Mistral AI’s Mixtral model is a well-known example; OpenAI’s newer GPT models are also widely believed to use some version of this approach, though the company has never officially confirmed it.<\/p>\n Diffusion is the tech at the heart of many art-, music-, and text-generating AI models. Inspired by physics, diffusion systems slowly “destroy” the structure of data \u2014 for example, photos, songs, and so on \u2014 by adding noise until there is nothing left. In physics, diffusion is spontaneous and irreversible \u2014 sugar diffused in coffee cannot be restored to cube form. But diffusion systems in AI aim to learn a sort of “reverse diffusion” process to restore the destroyed data, gaining the ability to recover the data from noise.<\/p>\n A GAN, or Generative Adversarial Network, is a type of machine learning framework that underpins some important developments in generative AI when it comes to producing realistic data \u2014 including (but not only) deepfake tools. GANs involve the use of a pair of neural networks, one of which draws on its training data to generate an output that is passed to the other model to evaluate. The two models are essentially programmed to try to outdo each other. The generator is trying to get its output past the discriminator, while the discriminator is working to spot artificially generated data. This structured contest can optimize AI outputs to be more realistic without the need for additional human intervention. Though GANs work best for narrower applications (such as producing realistic photos or videos), rather than general purpose AI.<\/p>\n Developing machine learning AIs involves a process known as training. In simple terms, this refers to data being fed in in order that the model can learn from patterns and generate useful outputs. Essentially, it is the process of the system responding to characteristics in the data that enables it to adapt outputs toward a sought-for goal \u2014 whether that is identifying images of cats or producing a haiku on demand. Training can be expensive because it requires lots of inputs, and the volumes required have been trending upwards \u2014 which is why hybrid approaches, such as fine-tuning a rules-based AI with targeted data, can help manage costs without starting entirely from scratch.<\/p>\n Fine-tuning refers to the further training of an AI model to optimize performance for a more specific task or area than was previously a focal point of its training \u2014 typically by feeding in new, specialized (i.e., task-oriented) data. Many AI startups are taking large language models as a starting point to build a commercial product but are vying to amp up utility for a target sector or task by supplementing earlier training cycles with fine-tuning based on their own domain-specific knowledge and expertise.<\/p>\n Transfer learning is a technique where a previously trained AI model is used as the starting point for developing a new model for a different but typically related task \u2014 allowing knowledge gained in previous training cycles to be reapplied. Transfer learning can drive efficiency savings by shortcutting model development. It can also be useful when data for the task that the model is being developed for is somewhat limited. However, models that rely on transfer learning to gain generalized capabilities will likely require training on additional data in order to perform well in their domain of focus.<\/p>\n Reinforcement learning is a way of training AI where a system learns by trying things and receiving rewards for correct answers \u2014 like training your beloved pet with treats, except the “pet” in this scenario is a neural network and the “treat” is a mathematical signal indicating success. Unlike supervised learning, where a model is trained on a fixed dataset of labeled examples, reinforcement learning lets a model explore its environment, take actions, and continuously update its behavior based on the feedback it receives. This approach has proven especially powerful for training AI to play games, control robots, and, more recently, sharpen the reasoning ability of large language models. Techniques like reinforcement learning from human feedback, or RLHF, are now central to how leading AI labs fine-tune their models to be more helpful, accurate, and safe.<\/p>\nNeural Network<\/h3>\n
Deep Learning<\/h3>\n
Compute<\/h3>\n
Model Types and Architecture<\/h2>\n
Large Language Model (LLM)<\/h3>\n
Mixture of Experts (MoE)<\/h3>\n
Diffusion Models<\/h3>\n
Generative Adversarial Network (GAN)<\/h3>\n
Training, Learning, and Optimization<\/h2>\n
Training<\/h3>\n
Fine-Tuning<\/h3>\n
Transfer Learning<\/h3>\n
Reinforcement Learning and RLHF<\/h3>\n
Distillation<\/h3>\n