Hierarchical Information Embeddings With Neural ODEs for Personalized Federated Learning

Abstract

Personalized federated learning (PFL) plays a pivotal role in ensuring efficient privacy preservation and secure collaborative learning. However, PFL faces significant challenges due to data heterogeneity and device diversity. To enhance personalization and robustness in PFL, we propose a novel model called FedNODE, which leverages hierarchical embeddings. FedNODE incorporates personalized, pseudo-generic, and fusion embeddings to facilitate hierarchical information representation. We utilize a hypernetwork based on neural ordinary differential equations (ODEs) within the server to generate backbone parameters for different clients, enabling the creation of personalized embeddings. Additionally, we introduce a pseudo-generic embedding based on a learnable vector to balance personalized and generic information. A neural ODE-based network follows the backbone module for each client, integrating personalized and pseudo-generic embeddings. To validate the efficacy of FedNODE, we conduct extensive evaluations across various classification datasets, encompassing diverse statistically heterogeneous settings and noisy scenarios. The results demonstrate that FedNODE achieves state-of-the-art performance.