Shifting Focus by Reliability: Geometric Fusion Modulation for Robust RGB-T Detection
Liuyin Shi, Chengqi Li, Wenbo He
Abstract
RGB-thermal (RGB-T) object detectors combine visible and thermal imagery for robust detection, but signal corruption in either stream can substantially degrade accuracy. Restoration-based methods incur significant latency and do not consistently improve downstream accuracy, raising a more direct question: rather than restoring corrupted inputs, can we efficiently adapt the detector to their varying signal quality? Conventional PEFT offers an efficient route, but its input-independent corrections cannot track diverse test-time corruptions. Our controlled corruption sweeps show that corruption types have similar effects near clean conditions but diverge increasingly with severity, motivating corruption-aware, per-input adaptation. We therefore propose PG-FM (Prompt-Guided Feature Modulation), which adapts RGB-T object detectors to the estimated corruption type and severity of each modality. Under controlled RGB-T corruption simulations, PG-FM preserves clean-set performance, improves mAP50 by up to 20.8 points on FLIR and 19.9 points on KAIST, and transfers without retraining to unmodified, naturally acquired external RGB-T datasets.