13 terms
Chips & Infrastructure
The hardware, data centers, and compute that make modern AI possible.
- CUDAnounNvidia's software platform for programming its GPUs, used by most AI frameworks
- data centernountrendinga large building full of servers, chips, cooling, and power equipment; AI data centers can draw as much electricity as a small…
- FLOPSnounfloating-point operations (per second): the basic unit for counting how much computing work a chip does, or how much a training…
- GPUnountrendinggraphics processing unit: a chip originally designed for video games that can do many calculations at the same time, which makes…
- hyperscalernounone of the few giant cloud companies that operate data centers on a massive scale and rent out computing power, including for AI
- KV cachenounmemory that stores intermediate attention results for text a model has already processed, so it doesn't have to recompute them…
- NPUnounneural processing unit: a small, power-efficient chip inside phones and laptops built to run AI features locally
- on-device AInounAI that runs directly on a phone, laptop, car, or other device instead of in the cloud, which improves privacy and works offline
- RAMageddonnountrending(informal) the shortage and sharp price rise of computer memory chips caused by AI data centers buying up supply, which makes…
- sovereign AInouna country's own AI capability, including data centers, models, and data, built so it does not depend entirely on foreign companies
- test-time computenountrendingextra computing power spent while answering a question, for example by letting the model think longer or try several answers…
- TPUnountensor processing unit: a chip designed by Google specifically for machine learning
- VRAMnounthe memory built into a GPU; how much a model needs largely decides which graphics card can run it
Advertisement