A team of researchers has introduced CLoVE (Clustering of Loss Vector Embeddings), a new algorithm for Clustered Federated Learning (CFL) that simplifies the process of grouping clients by data distribution without requiring near-optimal model initialization. According to the paper published on arXiv, CLoVE leverages client embeddings derived from model losses on client data, operating on the insight that clients in the same cluster share similar loss values while those in different clusters exhibit distinct loss patterns.
The algorithm iteratively identifies and separates clients from different clusters and optimizes cluster-specific models through federated aggregation. The authors—Bhatia, Randeep, Papadis, Nikos, Kodialam, Murali, Lakshman, TV, and Chakrabarty, Sayak—highlight three key advantages over existing CFL algorithms: simplicity, applicability to both supervised and unsupervised settings, and elimination of the need for near-optimal model initialization, making it more robust for real-world applications.
The paper establishes theoretical convergence bounds, showing that CLoVE can recover clusters accurately with high probability in a single round and converges exponentially fast to optimal models in a linear setting. Comprehensive experiments comparing CLoVE with a variety of CFL and generic Personalized Federated Learning (PFL) algorithms across different types of datasets and an extensive array of non-IID settings demonstrate that CLoVE achieves highly accurate cluster recovery in just a few rounds of training, along with state-of-the-art model accuracy.
| Feature | CLoVE | Typical CFL Algorithms |
|---|---|---|
| Initialization need | None | Often requires near-optimal model |
| Applicability | Supervised & unsupervised | Usually supervised only |
| Convergence speed | Exponential in linear setting | Varies |
| Cluster recovery accuracy | High in few rounds | Often slower |
For technology leaders evaluating privacy-preserving machine learning approaches, CLoVE offers a simpler, more robust method for personalization without centralizing sensitive data. The algorithm's theoretical guarantees and empirical performance make it a promising candidate for federated deployments where client data distributions vary significantly.