Mobility Helps Learning: Unsupervised Model Adaptation for Object Recognition via Movement
Yanan Ma, Yihang Tao, Zhengru Fang, Senkang Hu, Xianhao Chen, Yuguang Fang
Abstract
Movable agents, such as autonomous vehicles and robots, should be able to autonomously adapt their pre-trained object recognition models to new environments without human annotations. While existing test-time adaptation (TTA) methods address label-free model adaptation, they typically treat test data as independent snapshots, without constructively exploiting a rich, unique source of supervision: the significant variance in prediction quality as an agent observes the same object from different distances and viewpoints. To capitalize on this, we propose \texttt{MoCaFe}, a model-agnostic and hyperparameter-insensitive paradigm that leverages motion-induced prediction variance for unsupervised adaptation. Specifically, we first develop mobility-calibrated filtering of pseudo-labels, which constructs reliable pseudo-labels by fusing cross-view predictions using inverse-variance weights. The filtered pseudo-labels then guide the selection of ``hard samples'' for effective learning. To further suppress pseudo-label corruption, we introduce \texttt{p-MoCaFe}, an optional extension that uses a public dataset to assign weighting factors to data samples, yielding an estimator of the clean loss with bounded bias. Extensive experiments on autonomous driving (nuScenes, KITTI) and embodied-agent datasets show that \texttt{MoCaFe} significantly outperforms state-of-the-art TTA schemes, achieving up to approximately 20% accuracy gain.