Evaluating Early Multimodal Fusion in Discrete Latent World Models for Robot Navigation
1Kiel University, Germany · 2George Mason University, USA
We compare five multimodal fusion mechanisms (Shift, Scale, FiLM, Cross-Attention, and AdaLNZero) within Hydra, a discrete latent World Action Model for robot navigation, both offline and in 100 closed-loop physical robot trials under modality dropout. High-capacity operators excel offline but often degrade under physical sensor dropout, whereas restricting fusion to bounded element-wise scaling preserves the discrete latent structure and reaches up to 80% success under sensor loss.