TY - GEN
T1 - Crossing Domain Borders with Federated Few-Shot Adaptation
AU - Röder, Manuel
AU - Munch, Maximilian
AU - Raab, Christoph
AU - Schleif, Frank Michael
N1 - Publisher Copyright:
© 2024 by SCITEPRESS – Science and Technology Publications, Lda.
PY - 2024
Y1 - 2024
N2 - Federated Learning has gained significant attention as a data protecting paradigm for decentralized, clientside learning in the era of interconnected, sensor-equipped edge devices. However, practical applications of Federated Learning face three major challenges: First, the expensive data labeling process required for target adaptation involves human participation. Second, the data collection process on client devices suffers from covariate shift due to environmental impact on attached sensors, leading to a discrepancy between source and target samples. Third, in resource-limited environments, both continuous or regular model updates are often infeasible due to limited data transmission capabilities or technical constraints on channel availability and energy efficiency. To address these challenges, we propose FedAcross, an efficient and scalable Federated Learning framework designed specifically for real-world client adaptation in industrial environments. It is based on a pre-trained source model that includes a deep backbone, an adaptation module, and a classifier running on a powerful server. By freezing the backbone and the classifier during client adaptation on resourceconstrained devices, we enable the domain adaptive linear layer to solely handle target domain adaptation and minimize the overall computational overhead. Our extensive experimental results validate the effectiveness of FedAcross in achieving competitive adaptation on low-end client devices with limited target samples, effectively addressing the challenge of domain shift. Our framework effectively handles sporadic model updates within resource-limited environments, ensuring practical and seamless deployment.
AB - Federated Learning has gained significant attention as a data protecting paradigm for decentralized, clientside learning in the era of interconnected, sensor-equipped edge devices. However, practical applications of Federated Learning face three major challenges: First, the expensive data labeling process required for target adaptation involves human participation. Second, the data collection process on client devices suffers from covariate shift due to environmental impact on attached sensors, leading to a discrepancy between source and target samples. Third, in resource-limited environments, both continuous or regular model updates are often infeasible due to limited data transmission capabilities or technical constraints on channel availability and energy efficiency. To address these challenges, we propose FedAcross, an efficient and scalable Federated Learning framework designed specifically for real-world client adaptation in industrial environments. It is based on a pre-trained source model that includes a deep backbone, an adaptation module, and a classifier running on a powerful server. By freezing the backbone and the classifier during client adaptation on resourceconstrained devices, we enable the domain adaptive linear layer to solely handle target domain adaptation and minimize the overall computational overhead. Our extensive experimental results validate the effectiveness of FedAcross in achieving competitive adaptation on low-end client devices with limited target samples, effectively addressing the challenge of domain shift. Our framework effectively handles sporadic model updates within resource-limited environments, ensuring practical and seamless deployment.
KW - Deep Transfer Learning
KW - Domain Adaptation
KW - Federated Learning
KW - Few-Shot Learning
KW - Resource Constraints
KW - Sporadic Model Updates
UR - http://www.scopus.com/inward/record.url?scp=85190693782&partnerID=8YFLogxK
U2 - 10.5220/0012351900003654
DO - 10.5220/0012351900003654
M3 - Conference contribution
AN - SCOPUS:85190693782
SN - 9789897586842
T3 - International Conference on Pattern Recognition Applications and Methods
SP - 511
EP - 521
BT - Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods
A2 - Castrillon-Santana, Modesto
A2 - De Marsico, Maria
A2 - Fred, Ana
PB - SciTePress
T2 - 13th International Conference on Pattern Recognition Applications and Methods, ICPRAM 2024
Y2 - 24 February 2024 through 26 February 2024
ER -