A new research preprint on arXiv proposes a framework called PRO (Projected Rehearsal Orchestration) and its extension PRO-MAX to address a critical flaw in federated class-incremental learning (FCIL): the degradation of generator replay when clients observe different label subsets and progress through tasks at different stages. According to the paper by Nguyen, Thinh T H; Doan, Khoa D; Binh T; Le-Phuoc, Danh; and Wong, Kok-Seng, existing FCIL methods often preserve old knowledge through input-space synthesis (generator replay), but this approach is fragile under heterogeneous task streams and difficult to transfer across modalities.
The Problem with Generator Replay in FCIL
Federated class-incremental learning becomes substantially harder when clients observe different label subsets, progress through tasks at different stages, and provide uneven supervision for the same semantic concepts. The authors note that existing FCIL methods that rely on input-space synthesis can be fragile and hard to transfer across modalities like image, text, and graph data. They evaluated all methods under the same warmup to remove external pretraining bias.
The PRO and PRO-MAX Solution
To alleviate these issues, the researchers propose PRO, a framework that replaces synthetic input replay with projected rehearsal orchestration. PRO maintains compact class-level projected memories on the server and allows clients to perform balanced pseudo multi-task training over current examples and old projected memories. For stronger representation drift, they introduce PRO-MAX, which augments PRO with neighborhood-weighted memory alignment while preserving the same server-light principle: the server only aggregates model updates and memory statistics. This design avoids heavy server-side computation.
The key distinction from earlier work is that PRO does not rely on generative models for replay, which can degrade under heterogeneous conditions. Instead, it uses projected memories that stay aligned with the evolving representation.
Benchmark Results and Key Insights
The authors benchmarked PRO and PRO-MAX across image, text, and graph benchmarks. In heterogeneous streams, the methods improved retention and final utility compared to baselines, while remaining competitive in homogeneous FCIL. Even when baselines were given expanded replay budgets, they degraded under supervision imbalance and stage misalignment. This leads to a critical finding:
"replay quantity alone does not resolve replay-quality failures"
Additional weak-task diagnostics showed that larger replay mismatch is associated with larger downstream degradation. In contrast, PRO kept projected memories better aligned with the evolving representation, mitigating quality failures.
The paper presents a comparison of methods in the following table:
| Method | Replay Type | Server Load | Alignment Mechanism |
|---|---|---|---|
| Existing FCIL | Input-space synthesis (generator replay) | Higher (generation) | None |
| PRO | Projected rehearsal orchestration | Low (memory stats only) | Balanced pseudo multi-task training |
| PRO-MAX | Projected rehearsal with neighborhood weighting | Low (memory stats only) | Neighborhood-weighted memory alignment |
These results highlight that the proposed methods address the core challenge of replay quality in heterogeneous federated learning settings, offering a more robust approach for incremental learning across distributed clients with non-IID data.