AI Engineering Glossary

The practice of structuring content so AI answer engines and agents (ChatGPT, Claude, Perplexity, Google AI Overviews, Microsoft Copilot) cite it when answering user questions. The optimization target is citation inside an AI-generated answer, not a click.

An AI system with an outer reasoning loop where the model decides what to do next — including calling tools, branching, looping, and self-correcting. Agents handle multi-step tasks that fixed chains cannot complete reliably.

Persistent state across an agent's interactions, divided into four tiers: working (current task), episodic (past session summaries), semantic (extracted facts), and procedural (learned routines and policies).

A legal instrument under HIPAA that makes a vendor handling PHI on a covered entity's behalf a 'business associate' with defined obligations. Without an unbroken BAA chain, a HIPAA-covered AI deployment is non-compliant regardless of technical posture.

A high-dimensional numeric representation of text, images, or other media produced by an embedding model. Embeddings of similar content sit close in vector space, enabling semantic search and clustering.

An automated test infrastructure for AI systems comprising a golden set of scenarios, an LLM-as-judge rubric scorer, production trace replay, and quarterly human calibration. The minimum production-grade quality control.

The European Union's risk-tiered AI regulation, fully applicable from August 2026. Imposes substantive obligations on high-risk AI systems (HR, credit scoring, critical infrastructure, public services) including documentation, transparency, and human oversight.

Google's framing for content quality. Maps cleanly onto AI engine source-selection signals: identified authors, methodology disclosure, original observations, and verifiable affiliations.

The process of adjusting a pre-trained model's weights (or adding an adapter) so that its default behaviour shifts toward patterns in a smaller training dataset. Useful for style, format, and decision rules — not for teaching new facts.

A synonym for AEO. The practice of structuring web content so generative AI search engines surface and cite it. Key levers are answer-first structure, FAQPage schema, evidence density, and crawl-friendliness.

A metric measuring the proportion of factual claims in an AI-generated response that are supported by retrieved evidence. Production targets typically sit at 0.95 or above on a rolling 7-day window.

Runtime controls that enforce policy on AI inputs and outputs — prompt injection defenses, PII detection, toxicity scoring, citation requirement enforcement, and pre-action approval for destructive tools.

Any generated claim that is not supported by retrieved evidence or that contradicts ground truth. Types include intrinsic (contradicts retrieved passage), extrinsic (adds facts not in any passage), and citation fabrication (invents a source).

A retrieval pattern combining dense vector search and sparse keyword search (BM25), usually fused via Reciprocal Rank Fusion. Hybrid search consistently outperforms pure dense or pure sparse on heterogeneous corpora.

US federal law governing protected health information (PHI). HIPAA-compliant AI requires a signed BAA with every vendor handling PHI, encryption in transit and at rest, audit logging at request granularity, and six-year log retention.

A neural network trained on broad text corpora that generates language conditioned on a prompt. Production LLMs in 2026 range from sub-3B parameter edge models to frontier models above 400B parameters.

A framework for stateful, multi-actor AI applications built around cyclic graphs with explicit state, persistable checkpoints, and human-in-the-loop interruption. The strongest default for production agentic systems in 2026.

A parameter-efficient fine-tuning technique that adds small low-rank matrices to a pre-trained model's weights, drastically reducing training cost and adapter size. Standard for adapting open-weight LLMs in 2026.

A pattern where one LLM scores another's outputs against a rubric. Reliable when the judge is at least as capable as the model under test, the rubric is specific, and the judge is calibrated periodically against human labels.

A proposed convention (by Jeremy Howard) for a markdown digest of a website's content placed at the root URL, intended to give LLMs and AI agents a clean structured overview of the site for grounding and citation.

An AI architecture with multiple specialised agents collaborating on a task — for example research, writing, fact-checking, and publishing. Common frameworks include LangGraph, CrewAI, and AutoGen.

A pattern where a small model classifies query difficulty and routes to small, medium, or large models accordingly. Typically cuts total LLM spend 40–60% with no measurable quality loss when the classifier is well-tuned.

An open standard for connecting AI assistants to data sources and tools through a uniform interface. MCP servers expose resources, prompts, and tools that any MCP-compatible client can consume.

A provider feature that caches the prefill of stable prompt prefixes and charges 50–90% less for cached tokens. Materially reduces both cost and TTFT for production workloads with consistent system prompts.

An attack where malicious instructions in user input or retrieved content override the model's intended behaviour. Defences include input validation, output sanitisation, isolated tool sandboxes, and structured output enforcement.

A Next.js feature combining static prerendering of the page shell with dynamic streaming of interactive components. Delivers LCP under 1.2s even when AI TTFT is 600ms or more.

A compression technique that reduces the numeric precision of model weights (e.g. FP16 to INT8 or INT4) to cut memory footprint and accelerate inference, with small accuracy trade-offs.

An architecture where a large language model generates answers grounded in documents retrieved at inference time, rather than relying solely on its training data. A typical RAG system has four stages: embedding, retrieval, optional reranking, and grounded generation.

A model that re-scores retrieved candidates by jointly encoding the (query, document) pair, producing more accurate relevance scores than the bi-encoder used in initial retrieval. Standard rerankers in 2026 include BGE-reranker-v2-m3 and Cohere Rerank 3.

A RAG architecture where every component — embedding model, vector store, retriever, and LLM — runs inside infrastructure the organisation legally controls. No request, embedding, or document ever leaves the customer-owned VPC, region, or air-gapped data center.

A cache that stores responses keyed by the embedding of the query, returning a cached response when a new query is semantically similar. Cuts hot-query latency to sub-50ms and reduces inference cost on repeat queries.

The 2020 Court of Justice of the European Union ruling that invalidated the EU-US Privacy Shield and made every trans-Atlantic personal data transfer a legal-risk decision rather than a default. A primary driver of sovereign AI adoption in the EU.

An AICPA framework demonstrating operational controls over Security, Availability, Processing Integrity, Confidentiality, and Privacy. Type I is point-in-time; Type II requires twelve months of operating-effectiveness evidence.

React components that render exclusively on the server and ship zero JavaScript to the client. The largest single lever for bundle size and time-to-interactive in modern React applications.

A schema.org property identifying the CSS selectors on a page most suitable for voice assistants to read aloud. Telegraphs to AI engines which content is the cleanest extractable answer.

The latency between sending a prompt to an LLM and receiving the first character of the response. Below ~400ms feels instant; above ~1s users perceive the system as 'thinking too hard.' TTFT is the dominant UX metric for conversational AI.

The capability of an LLM to call structured external functions — search, database queries, code execution, API calls — as part of its response, enabling agents to take actions beyond text generation.

A storage system optimised for high-dimensional vector embeddings with approximate nearest neighbour search. Major options in 2026 include Pinecone (managed), Qdrant (self-hosted), Weaviate (hybrid search), and pgvector (Postgres extension).