A two-party scheme that lets a client run inference on a LoRA fine-tuned large language model without revealing its prompt to the server and without the server revealing its private adapter weights to the client. The model is split so the public open-source base weights run in plaintext on the client and only the private LoRA matrices stay server-side, which keeps the expensive nonlinear operations (softmax, GELU, LayerNorm) off the encrypted path. The Private Linear Layer (PLL) is the construction that hides the server's weights: a linear map is replaced by a functionally equivalent one whose parameters are masked so that recovering them from query-response pairs reduces to the Learning With Errors problem. Composing PLL with fully homomorphic encryption over the channel gives correct inference that protects both sides at once, measured at 1.61 seconds per token on ChatGLM2-6B.
A secure inference scheme for a large language model that has been fine-tuned with LoRA, designed so that neither the user's input prompt nor the server's private fine-tuning weights are exposed to t…