← All writing

Attacks on Federated Learning: The Membership Inference Threat

Mar 23, 2026

PrivacyMLSecurity

Federated Learning (FL) promises privacy by keeping data on your device, but it isn't bulletproof. Attackers can analyze model updates or outputs to run several types of attacks, including Membership Inference Attacks (MIAs), which try to determine whether a specific individual's data was used to train the model. Adding explainability through Counterfactual Explanations makes these attacks even more effective. During my thesis, I found that adding explanations increases attack success by up to 9%, but applying the right privacy-preserving techniques can cut this risk sharply without destroying accuracy.

Imagine this: you participate in a decentralized medical study using an app on your smartphone. The app trains a diagnostic model locally on your health or fitness records and only sends the mathematical updates -not your raw data -back to a central server. You feel secure knowing your information never left your phone.

A few weeks later, an attacker intercepts the model updates. By analyzing certain parameters of the model and its behavior, they deduce whether your specific health record was part of the training data. The fact that you have a specific medical condition is now exposed. This is a known, documented vulnerability called a Membership Inference Attack (MIA).

The Illusion of Perfect Privacy

Federated Learning protects data at rest. By bringing the computation to the data, it ensures raw datasets are never sent to a central point of vulnerability. However, FL creates indirect vulnerabilities through the aggregated global model.

The training parameters, the weights and gradients sent back and forth between your device and the central server, are a leakage vector. Models memorize data, especially in "low-data regimes," where only a handful of training examples exist.

Attackers can exploit this using two main approaches: Blackbox Attacks: The attacker only sees the final predictions and confidence scores the model outputs. By checking if the model is overly confident about a specific prediction, they can guess if that data point was in the training set. Whitebox Attacks: The attacker has access to the internal parameters and gradients of the model. These are far more dangerous. In whitebox attacks, the average success rate is 8% to 13% higher than in blackbox attacks, reaching up to an 87.4% success rate in some scenarios.

The Explainability Trap and New Attack Vectors

As I covered in past articles, new laws like the EU AI Act require AI to be transparent. Techniques like Counterfactual Explanations (CF) help by telling users what needs to change to get a different outcome ("If your income was €5,000 higher, your loan would be approved").

But there is a catch. The more an AI model explains about its decision boundaries, the more information it gives to an attacker. Counterfactual generators open richer information channels that amplify Membership Inference and open the door to a wider range of attacks:

  • Linkage Attacks: Attackers combine explanations and model outputs with auxiliary public datasets in order to re-identify individuals or infer private attributes.
  • Model Extraction Attacks: Adversaries can attempt to "steal" the ML model. By repeatedly querying the system and analyzing the predictions and explanations, they can train a copycat model that behaves identically.
  • Generative Model Attacks: With the rise of advanced generative techniques, it has been shown that models can inadvertently memorize and reproduce exact training examples. Attackers can exploit this to extract rare or unique data points.
  • Data Reconstruction Attacks: Even more intrusively, attackers can exploit gradients or model queries to probabilistically reconstruct authentic samples or entire datasets directly from the model updates.

During my research, I quantified this increased risk specifically for MIAs. The addition of Counterfactual Explanations generally increased the success rate of attacks: Blackbox MIAs became 6.5% more successful on average. Whitebox MIAs became 9.0% more successful on average.

We face a fundamental tension: the most accurate and explainable models are simultaneously the most vulnerable to privacy attacks.

Conclusion

Federated Learning is a massive leap forward for data privacy, but it is not a silver bullet. As we push to make AI more transparent and explainable, we are inadvertently handing attackers the exact tools they need to break that privacy through Membership Inference, Linkage and Data Reconstruction attacks.

Secure AI isn't about applying maximum privacy to everything, which would render our models useless. It is about understanding how these threats evolve and engineering targeted defenses against them. Organizations must balance three competing demands: model accuracy, auditability through explanations and the confidentiality of the underlying training data.

References

  • [1] Molinari, L., "Privacy Leakage of Counterfactual Explanations in Federated Learning," Master's Thesis, USI/UniMiB, 2025.
  • [2] Shokri, R., et al., "Membership Inference Attacks Against Machine Learning Models," S&P, 2017.
  • [3] Steinke, T., et al., "Privacy Auditing with One (1) Training Run," 2023.
  • [4] Goethals, S., et al., "The Privacy Issue of Counterfactual Explanations: Explanation Linkage Attacks," 2023.
  • [5] European Commission, "Regulation (EU) 2016/679 (General Data Protection Regulation)," 2016.
  • [6] European Commission, "Regulation (EU) 2024/1689 (Artificial Intelligence Act)," 2024.