Skip to content
AI

Inference

Inference is the moment a trained model generates an answer from new input, as opposed to the training phase.

Training sets the model's parameters once, or nearly so. Inference is every following call, such as a question asked to a chatbot, that uses those parameters without changing them.

Inference speed and cost are what a company pays each time it uses a model, unlike training cost, which the model provider pays once.