Fluorescein angiography (FFA) is a critical diagnostic tool for assessing retinal vascular abnormalities, but its acquisition is invasive and not always feasible. In contrast, color fundus photography (CFP) is non-invasive and widely accessible, motivating research into CFP-to-FFA synthesis. However, prior works rely solely on CFP surface texture, fundamentally limiting the ability to reconstruct functional vascular information and subtle pathological changes.
To address this, a team of researchers led by Ma proposed a novel framework that synthesizes FFA from CFP with structural guidance provided by optical coherence tomography (OCT), according to a paper published on arXiv.
Dataset and Alignment
The team constructed a multi-modal retinal imaging dataset with paired CFP, FFA, and OCT from 3,676 patient eyes—described as the first tri-modally aligned dataset in retinal imaging. This dataset bridges a critical gap by providing co-registered information across three modalities.
Technical Approach
The framework introduces two key components:
- Spatially Aligned Cross-Modal Fusion (SACMF) module: Projects depth-resolved OCT features onto the fundus plane and injects them into the CFP encoder via adaptive layer normalization.
- Token-wise Cross-Modality Alignment (TCMA): A token-level contrastive learning strategy that explicitly aligns CFP and FFA representations at corresponding spatial positions.
These modules enable the model to propagate structural guidance from OCT scans to improve FFA synthesis.
Performance and Clinical Potential
The method achieves superior synthesis performance compared to state-of-the-art methods. Moreover, extensive experiments demonstrate that the FFA images synthesized by this approach bring greater improvements in downstream disease diagnosis performance than existing methods, highlighting the clinical potential as a non-invasive decision-support tool in routine workflows.
| Component | Function |
|---|---|
| SACMF module | Projects depth-resolved OCT features onto fundus plane, injects into CFP encoder |
| TCMA | Token-level contrastive learning to align CFP and FFA representations |
| Tri-modal dataset | Paired CFP, FFA, OCT from 3,676 eyes |
The code for the framework is publicly available via a link in the paper.
This work could enable wider access to FFA-like imaging without the invasiveness and risks of dye injection, potentially making retinal vascular assessment more routine in clinical settings.