Topic
heterogeneous
Mosaic: Data-Free Knowledge Distillation Framework Uses Mixture-of-Experts to Tackle Heterogeneous Federated Learning
Researchers propose Mosaic, a novel data-free knowledge distillation framework that leverages Mixture-of-Experts (MoE) to overcome model and data heterogeneity in federated learning. Mosaic trains local generative models to synthesize data, forms an MoE from client models, and distills it into a global model. Experiments show consistent outperformance over state-of-the-art approaches on image and multimodal benchmarks.
When Generator Replay Degrades: Projected Rehearsal Orchestration for Heterogeneous Federated Class-Incremental Learning
A new arXiv preprint introduces PRO and PRO-MAX, frameworks that replace synthetic input replay with projected rehearsal orchestration to address degradation in federated class-incremental learning (FCIL) when clients have heterogeneous label subsets and task stages. The methods improve retention and utility across image, text, and graph benchmarks, showing that replay quantity alone does not resolve quality failures.