← Research

Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactions

Caixin Kang, Yifei Huang, Liangyang Ouyang, Mingfang Zhang, Ruicong Liu, Yoichi Sato

Despite their advanced reasoning capabilities, state-of-the-art Multimodal Large Language Models (MLLMs) still lack a core component of human intelligence: the ability to "read the room" and assess deception in complex social interactions. To rigorously evaluate this limitation, we introduce Multimodal Interactive Deception Assessment (MIDA), a new task and multimodal dataset comprising synchronized video and text with verifiable ground-truth labels for every statement. We benchmark 12 state-of-the-art open- and closed-source MLLMs and identify a substantial performance gap: even powerful models such as GPT-4o struggle to reliably distinguish truthful statements from deceptive ones. Our analysis shows that current models have difficulty grounding language in multimodal social cues and modeling what others know, believe, or intend. To address these limitations, we propose a Social Chain-of-Thought (SoCoT) reasoning pipeline and a Dynamic Social Epistemic Memory (DSEM) module. Experimental results show that structured epistemic memory consistently improves deception assessment, while multimodal social-cue reasoning remains a challenging and important direction for future research.

PDF ↗

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace