Topic
foundation models
EEG Foundation Models Show Promise for Burst-Suppression Detection in ICU Without Patient-Specific Calibration
A new study on arXiv evaluates three EEG foundation models—REVE-base, LUNA-large, and LuMamba-Tiny—for automatic burst-suppression detection in ICU patients, finding REVE-base achieves the highest event-based F1-score (0.868) and reduces burst-per-minute error by 52.1% compared to a task-specific EEGNet baseline.
Reinforcement Learning Foundation Models: Synthetic MDPs Could Bridge the Gap
The paper by Zighem, Abdelrahman, and Vie argues that reinforcement learning (RL) lacks a foundation model equivalent to those for language and vision. They propose using synthetic Markov Decision Processes (MDPs), which are as feasible to generate as synthetic tabular data, and demonstrate with a Graph Attention Network trained entirely on synthetic MDPs that achieves competitive results without task-specific tuning.
DiverseDistill: New Knowledge Distillation Method Recovers Over 70% of Performance Gap Using Teacher Committees
Researchers propose DiverseDistill, a knowledge distillation framework that combines a large foundation model with domain-specific experts as a diverse committee. The method recovers 73–114% of the teacher-student performance gap on recommendation and vision tasks while requiring no parameter updates or architectural changes.
First Billion-Parameter Generative Foundation Model for Chest Radiography Achieves Expert-Level Synthesis Fidelity
Ribeiro et al. present the largest specialist generative foundation model for chest radiographs, with over 1.3 billion parameters. Trained on 1.2 million radiographs, the model supports controllable generation across demographics, views, and pathologies, advancing synthesis fidelity to clinical indistinguishability.
Study Finds Hybrid CNN-Clay Model Improves Landslide Detection Accuracy Over Baseline
A study evaluates Clay v1.5, a Geospatial Foundation Model, for pixel-level landslide segmentation on the Landslide4Sense benchmark. The hybrid U-Net + Clay model with two-stage LoRA achieves a test F1 of 64.5%, outperforming both the Clay-only backbone and a standard U-Net baseline.
Boosting Knowledge Graph Foundation Models via Enhanced Negative Sampling
Researchers propose KMAS, an adaptive negative sampling approach that enhances knowledge graph foundation models (KGFMs) by generating hard negative triples from relation embeddings. The method dynamically adjusts the ratio of hard negatives during training, improving performance across 44 datasets without significant extra time or memory.
TS-Memory: A Plug-and-Play Memory Adapter for Time Series Foundation Models
TS-Memory, a lightweight memory adapter proposed by researchers, addresses distribution shift in time series foundation models. It uses Parametric Memory Distillation to combine the benefits of parametric adaptation and non-parametric retrieval without their drawbacks, achieving consistent forecasting improvements with minimal overhead.
New EEG Benchmark Promises Standardized Evaluation of Foundation Models
A new benchmark called EEG-FM-Bench aims to standardize evaluation of electroencephalography foundation models (EEG-FMs). It integrates 14 datasets across 10 paradigms and provides tools for gradient and representation analysis. Early experiments reveal critical insights about multi-task learning, pre-training efficiency, and model scaling.
Cough Regression Benchmark Reveals Trade-Offs in Respiratory Acoustic Foundation Models
A new benchmark from researchers at NC State evaluates five respiratory acoustic foundation models on cough regression tasks—predicting age, BMI, and disease probability from cough audio. The study reveals that smaller MLP heads often outperform linear probes, but full-MLP heads overfit on small clinical data. HeAR and M2D+Resp achieve near-full performance with only 50 samples, while OPERA models require 400. Cross-dataset transfer is asymmetric, with large diverse datasets generalizing better to small clinical populations.
FusionRS Dataset Advances Dual-Modal Vision-Language AI for Remote Sensing
Researchers introduced FusionRS, the first large-scale RGB-infrared-text dataset for dual-modal vision-language learning in remote sensing. The dataset pairs RGB and infrared images with scene and IR-aware captions, enabling models to achieve better alignment and retrieval than RGB-only approaches.
Mutual Distillation of Dual Foundation Models Achieves State-of-the-Art PET/CT Segmentation with Only 5 Labeled Cases
Researchers propose MuDuo, a mutual distillation framework that leverages two foundation models (SAM-Med3D for CT, SegAnyPET for PET) to distill knowledge into a lightweight student network for semi-supervised PET/CT segmentation. Achieving state-of-the-art performance on the AutoPET dataset with only 5 labeled cases, the approach eliminates manual prompts and maximizes unlabeled data utility.
New Research Reveals Spatial Audio Foundation Models Rely on Spectro-Temporal Interference Rather Than True Phase Encoding
Researchers evaluated nine audio models using a binaural masking level difference benchmark, finding that general-purpose binaural SSL models lack true phase sensitivity and instead rely on spectro-temporal interference textures, while dedicated spatial SSL models perform comparably to analytical baselines.
A Theoretical Roadmap to Fuse Foundation Models and Knowledge Graphs
A new theoretical paper formalizes the 'Impedance Mismatch' between Foundation Models and Knowledge Graphs, arguing that current approaches like RAG are superficial. The authors propose a roadmap including Structured Residual Streams, Vector Symbolic Architectures, and Orthogonal Subspace Editing for true semantic fusion.
GeoRoPE: Ground-Aware Rotary Adaptation Enhances Remote Sensing Foundation Models
A new research paper introduces GeoRoPE, a ground-aware rotary adaptation method for remote sensing foundation models. It addresses scale mismatch by recalibrating token-level positional interactions, improving cross-resolution robustness and scale-sensitive representation learning. The method is parameter-efficient and compatible with existing models.