Machine learning (ML) systems are increasingly being deployed in high-stakes domains such as healthcare, finance, and security, where they often handle sensitive and proprietary data. While these models provide significant utility, they are inherently vulnerable to privacy attacks, in which adversaries attempt to infer confidential information from the system’s outputs.
One of the long known trait of Deep Learning has been Memorization. It is the condition when the model, instead of picking up the general patterns, begins to memorize each example it cannot correctly classify. It can cause lots of problems to the model like, Privacy vulnerability, hampering the model performance. Detecting if the model has been memorizing certain training instances can help in improving the model performance and reduce it’s vulnerability to privacy attacks.