DeML OS - 2026-02-07

DeML OS Daily DeML OS 最新前沿分析

Explore Frontier

02.07

2026

Sat

📄

Paper

Privacy-Preserving LLM Inference in Practice: A Comparative Survey of Techniques, Trade-Offs, and Deployability https://eprint.iacr.org/2026/105

Davide Andreoletti Confidential Inference

In the paper, Andreoletti et al. survey privacy-preserving LLM inference techniques, comparing TEE, crypto-augmented designs, and FHE in terms of trust, scalability, and deployment readiness under a strong end-to-end privacy model.

Notes

Defines a strong privacy model where only clients can access prompts and outputs.
Reviews PET-based approaches for LLM inference, including handling non-linear layers and autoregression.
TEE-based systems are the most deployable today but rely on hardware trust.
Crypto-augmented designs reduce hardware trust at higher computational cost.
FHE is positioned as a principled long-term solution for confidential inference.
Highlights trade-offs between security, performance, and deployability.

Collected by @icerdesign

DeML OS Q & A 问答

Deep Dive 💬

02.07

2026

Sat

😇

Why are non-linear layers in Transformers especially challenging for private inference?

Non-linear functions such as GELU and Softmax are hard to implement efficiently under encryption or restricted execution. They often require approximations, interaction, or extra trust assumptions, making them major performance bottlenecks.

😎

😊

Why are TEE-based solutions considered the most deployable today?

TEEs can run full models at near-native performance and support autoregressive decoding with low engineering complexity. The main cost is reliance on hardware trust and side-channel defenses rather than computation.

😎

🤓

Why is autoregressive decoding a key differentiator for private inference systems?

Autoregressive decoding requires token-by-token dependency, which is extremely costly under interactive or high-latency cryptographic schemes. Efficient decoding support largely determines real-world applicability.

😎

Prompted by @icerdesign