35 terms
Chips & Infrastructure
The hardware, data centers, and compute that make modern AI possible.
- agentic workloadnountrendingthe pattern of computing work produced when an AI agent pursues a goal through many dependent model calls, tool calls, and state…
- AI compute gridnountrendingan interconnected system that pools AI computing capacity from multiple data centers, clouds, chip types, or organizations and…
- AI factorynountrendinga purpose-built, full-stack computing environment that combines data pipelines, accelerated hardware, networking, storage, power…
- AI infrastructurenountrendingthe integrated hardware, networking, data, software, and physical facilities used to train, deploy, scale, and operate AI systems
- AI PCnountrendinga personal computer designed to run artificial-intelligence workloads locally, typically by combining a CPU and GPU with a…
- AI-native networknountrendinga telecommunications or computer network designed with AI as a core part of its architecture and operation, using learning…
- batch inferencenounrunning a trained AI model on a collected group of inputs as a scheduled or asynchronous job when individual results do not need…
- chip export controlsnountrendinglegal restrictions on exporting, reexporting, or transferring advanced processors, semiconductor equipment, software, or related…
- continuous batchingnounan inference-serving technique that adds new requests to and removes completed requests from a model's processing batch between…
- CUDAnounNvidia's software platform for programming its GPUs, used by most AI frameworks
- data centernountrendinga large building full of servers, chips, cooling, and power equipment; AI data centers can draw as much electricity as a small…
- data residencynouna requirement or arrangement specifying the geographic region in which data is stored, processed, or otherwise handled
- FLOPSnounfloating-point operations (per second): the basic unit for counting how much computing work a chip does, or how much a training…
- goodputnounthe rate or proportion of useful machine-learning work completed after time lost to idle resources, scheduling delays, stalls…
- GPUnountrendinggraphics processing unit: a chip originally designed for video games that can do many calculations at the same time, which makes…
- GPU-backed loannountrendinga loan or debt facility secured partly or primarily by graphics processing units and the computing equipment built around them…
- GPU-pooradjectivehaving too little access to high-performance graphics processors for the desired AI training or inference work; used in contrast…
- HBMnouna type of three-dimensionally stacked memory that uses a very wide connection to move data rapidly between memory and a nearby…
- hyperscalernounone of the few giant cloud companies that operate data centers on a massive scale and rent out computing power, including for AI
- KV cachenounmemory that stores intermediate attention results for text a model has already processed, so it doesn't have to recompute them…
- neocloudnountrendinga cloud-computing provider built primarily around high-performance GPU capacity and AI workloads, typically offering clusters…
- NPUnounneural processing unit: a small, power-efficient chip inside phones and laptops built to run AI features locally
- on-device AInounAI that runs directly on a phone, laptop, car, or other device instead of in the cloud, which improves privacy and works offline
- orbital data centernountrendinga proposed data center for AI computing placed in orbit around the Earth, where it could run on constant solar power and shed…
- prefill-decode disaggregationnountrendingan architecture for serving language models that runs the compute-heavy prefill stage, which processes the input prompt, on…
- RAMageddonnountrending(informal) the shortage and sharp price rise of computer memory chips caused by AI data centers buying up supply, which makes…
- sovereign AInouna country's own AI capability, including data centers, models, and data, built so it does not depend entirely on foreign companies
- sovereign computenountrendingcomputing capacity that a country or organization can reliably access and control under its chosen jurisdiction and governance…
- speculative decodingnountrendinga technique for speeding up autoregressive model inference in which a faster draft process proposes several tokens and a larger…
- test-time computenountrendingextra computing power spent while answering a question, for example by letting the model think longer or try several answers…
- TinyMLnounthe field of running machine-learning models directly on extremely low-power, resource-constrained hardware such as…
- token factorynountrendingan AI inference operation that turns GPU compute into token-metered model output by combining model hosting, optimized serving…
- TPUnountensor processing unit: a chip designed by Google specifically for machine learning
- unified memorynouna memory architecture or programming model that lets a CPU and one or more accelerators access a shared address space or memory…
- VRAMnounthe memory built into a GPU; how much a model needs largely decides which graphics card can run it
Advertisement