J
jeremy
8 minutes
Sampling lets a server request a model completion through the connected client, moving inference cost, key custody, and model choice to the client side.