Fully Homomorphic Encryption (FHE) enables a server to run a language model directly on encrypted user prompts, yet current solutions remain prohibitively slow. Ciphertexts natively support only addition, multiplication, and rotation; multiplications can be composed to a limited depth before a costly bootstrapping step is required to continue computation. Consequently, every non‑linearity must be approximated by an iterative method, and each iteration consumes multiplications. More iterations improve precision but exhaust depth faster and trigger additional bootstraps, which dominate latency. Existing pipelines fix the iteration count uniformly across the model, ignoring the varying error tolerance of different layers.
We introduce Homomorphic Encryption‑Aware Training (HEAT), a fine‑tuning technique that makes the per‑nonlinearity iteration counts learnable and co‑adapt them with model weights during training. HEAT treats iteration counts as differentiable hyper‑parameters optimized directly against the task objective, allowing the model to compensate for approximation errors at inference time without architectural changes or retraining from scratch.
In encrypted GPT‑2 decoding experiments, HEAT reduces the average iteration count by $3.1\times$, bootstraps by $1.6\times$, and end‑to‑end latency by $1.4\times$, while achieving higher decode agreement than a calibrated baseline.
Review: By jointly learning iteration budgets and weights, HEAT dramatically cuts the computational overhead and latency of homomorphic inference, paving a more practical path for deploying encrypted language models.