A study about the performance of federated learning under out-of-distribution data groups

College of Letters & Science, University of California, Santa Barbara, USA

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

Federated learning (FL) is a promising machine learning paradigm that focuses on sending data to the client, then using the local unique data to train the local model. Only the model updates are sent back to be aggregated into a global model; this action can efficiently protect the data privacy and security. However, developing the FL model in the real world is still unreliable due to the problem that the real-world dataset always contains both desired in-distribution data and unexpected out-of-distribution (OOD) data. Current FL researchers typically use prefabricated artificial data sets to conduct a series of experiments and obtain positive results, yet most of these datasets are in-distribution data. In this work, we propose a light weight Client-Side OOD Filtering method tailored for real-world FL compatibility. This method integrates seamlessly with standard FL frameworks by adding a simple pre-processing step on the client side, without modifying server-side aggregation logic. The core design includes two key components: (1) A lightweight ID classifier that quantifies the alignment between local data and ID distribution with soft max probability. (2) An adaptive threshold mechanism that retains data with probability ≥ IID data for regional training and filters out low probability data.