- When we talk about Privacy testing in machine learning, if we consider testing for all possible attacks and vulnerabilities and all privacy protection mechanisms, the scope would just be too huge.
- For this reason, when testing privacy in ML, researchers have mostly focused on privacy protection methods that are the most popular and narrowed their focus to a single one
- The most popular privacy protection technique currently used is Differential privacy. It mainly provides protection for the members of the Training dataset used for model. Since, the privacy protection of the users whose data was used to train the model is of highest priority, Differential privacy testing has also received the highest priority.
When we are talking about Privacy testing in differential privacy, the main approaches that have currently been developed are:
- Programming Language based testing methods ( white box): They use language based methods to automatically verify mechanism’s privacy level. While, they are particularly useful in formally verifying if the implementation correct or buggy, they still have a major disadvantage of not being universal. There are lots of differential privacy implementations and different libraries in use right now with different variations and we would need to design test cases for each of them separately.
- Probabilistic Testing Methods ( mainly Black Box): They use statistical tools and is based on sampling the mechanism’s inputs and outputs. Specifically, they focus on estimating the lower-bound of the privacy parameter of Differential privacy. In other words, they determine the maximum level of privacy that the implementation will offer with statistical confidence. This line of work is more researched than programming language based testing probably because of the universality of testing.
The above two methods considers the implementation as (Algorithm + Differential Privacy). In this the algorithm could be something like Noisy Max or a machine learning model.
Collection of Relevant Papers
Latest Research Papers on Privacy Attacks(2020-Present)
Relevant Resources
Relevant Survey Papers
Black Box Differential Privacy Evaluation Techniques