Let Them Drop: Scalable and Efficient Federated Learning Solutions Agnostic to Stragglers - 3IA Côte d’Azur – Interdisciplinary Institute for Artificial Intelligence
Communication Dans Un Congrès Année : 2024

Let Them Drop: Scalable and Efficient Federated Learning Solutions Agnostic to Stragglers

Résumé

Secure Aggregation (SA) stands as a crucial component in modern Federated Learning (FL) systems, facilitating collaborative training of a global machine learning model while protecting the privacy of individual clients' local datasets. Many existing SA protocols described in the FL literature operate synchronously, leading to notable runtime slowdowns due to the presence of stragglers (i.e. latearriving clients). To address this challenge, one common approach is to consider stragglers as client failures and use SA solutions that are robust against dropouts. While this approach indeed seems to work, it unfortunately affects the performance of the protocol as its cost strongly depends on the dropout ratio and this ratio has increased significantly when taking stragglers into account. Another approach explored in the literature to address stragglers is to introduce asynchronicity into the FL system. Very few SA solutions exist in this setting and currently suffer from high overhead. In this paper, similar to related work, we propose to handle stragglers as client failures but design SA solutions that do not depend on the dropout ratio so that an unavoidable increase on this metric does not affect the performance of the solution. We first introduce Eagle, a synchronous SA scheme designed not to depend on the client failures but on the online users' inputs only. This approach offers better computation and communication costs compared to existing solutions under realistic settings where the number of stragglers is high. We then propose Owl, the first SA solution that is suitable for the asynchronous setting and once again considers online clients' contributions only. We implement both solutions and show that: (i) in a synchronous FL with realistic dropout rates (taking potential stragglers into account), Eagle outperforms the best SA solution, namely Flamingo, by ×4; (ii) In the asynchronous setting, Owl exhibits the best performance compared to the state-of-the-art solution LightSecAgg .

Fichier principal
Vignette du fichier
3664476.3664488.pdf (1.03 Mo) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04742784 , version 1 (21-10-2024)

Licence

Identifiants

Citer

Riccardo Taiello, Melek Önen, Clémentine Gritti, Marco Lorenzi. Let Them Drop: Scalable and Efficient Federated Learning Solutions Agnostic to Stragglers. ARES 2024 - 19th International Conference on Availability, Reliability and Security, Jul 2024, Vienna, Austria. pp.1-12, ⟨10.1145/3664476.3664488⟩. ⟨hal-04742784⟩
199 Consultations
61 Téléchargements

Altmetric

Partager

More