QUOKKA: Faster Secure Neural Network Inference with Early-Exit Technology

Secure Neural Network Inference (SNNI) protocols, vital for privacy-preserving AI, face substantial computational and communication overhead. Dynamic Early-Exit (EE) networks could help decrease the overhead, but existing SNNI protocols do not support such networks. We introduce QUOKKA, the first system to enable SNNI for confidence-based EE neural networks using secure Multi-Party Computation. QUOKKA addresses the challenges of dynamic decision-making and sensitive intermediary result handling in SNNI. Implemented with EENet and CrypTen, QUOKKA achieves 2–5× acceleration over traditional SNNI without a decrease in accuracy. Our findings demonstrate practical, highly efficient, and privacy-preserving SNNI for dynamic AI, paving the way for broader Machine-Learning-as-a-Service deployment.