Cross-modal inconsistency is a major challenge for reliable RGB-D anomaly detection, especially under few-shot settings where normal multimodal patterns are insufficiently represented. In this work, we study hallucination in a restricted perception-level sense, referring to unreliable anomaly responses induced by inconsistent or insufficiently grounded multimodal evidence. Unlike textual hallucinations in LLMs/VLMs, the phenomenon considered here occurs in multimodal perception, where conflicting RGB appearance and depth geometry may lead to false defect alarms, missed anomalies, or inaccurate localization maps.
To address this problem, we introduce Hallucination-Aware Hypergraph Memory (HA-HM), a framework that models high-order cross-modal consistency through a unified, learnable hypergraph. HA-HM integrates four synergistic components: a Learnable Hypergraph Constructor for adaptive multimodal grouping, a Consistency-Guided Memory Prototype Learner that stores high-fidelity normal prototypes, an Iterative Hypergraph Refiner for prototype-guided inconsistency correction, and a Hallucination Localization module for dense anomaly localization. Evaluations on the MVTec 3D-AD and Eyecandies RGB-D benchmarks demonstrate the effectiveness of HA-HM in few-shot multimodal anomaly detection and localization.
Under challenging 1-, 2-, and 4-shot conditions, HA-HM achieves consistent improvements over existing training-based and training-free methods in image-level AUROC, pixel-level AUROC, and AUPRO. Ablation studies further validate the contribution of each component. These results suggest that explicitly modeling cross-modal consistency can improve the reliability of RGB-D anomaly detection systems.
School of Statistics and Data Science, Shanghai University of International Business and Economics, Shanghai, 201620, China Jiayi Liu, Xiyue Fan, Xin Peng, Yixin Xu & Yi Wang All authors have reviewed the manuscript and consent to its publication. No identifiable information regarding participants has been included. The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Extract — continue reading at the source.
