Exploring Hyper-Parameter Sensitivity for Improved Privacy and Utility in Differentially Private Models on Real-World Medical Data

Privacy is gaining an increasing concern in practical business scenarios, yet only a small fraction of production deployments of deep learning models is accounted for by differentially private models. This paper explores the potential of differentially private deep learning (DP-DL) models to preserve data privacy while achieving high performance, sometimes surpassing non-private models. Amid rising data privacy regulations and awareness, our study challenges the belief that privacy degrades model performance. Using deep learning models on the PIMA Indians Diabetes Dataset and the Breast Cancer Wisconsin Dataset, we fine-tuned hyper-parameters through a comprehensive grid search and implemented differential privacy with the Opacus library. Our findings reveal that DP-DL models can maintain or exceed the accuracy of non-private counterparts under specific conditions, providing a nuanced understanding of the balance between privacy and utility.