prompt caching

Definition of prompt caching

1. : reusing the already-processed beginning of a prompt across many requests, which cuts cost and speeds up responses when the same long instructions or documents are sent repeatedly

   • Prompt caching cut our bill by 70% because every request starts with the same manual.
   • Put the parts that never change at the top so prompt caching can work.