← Research

From Scene-Centric to Observer-Centric: Modeling Observer-Aware Relations for 3D Scene Graph Generation

Jingjun Sun, Chaowei Wang, Zhirui Liu, Jiaxu Tian, Ming Yang, Yaoxing Wang, Yan Di, Shan Gao

3D Scene Graph Generation (3DSGG) represents 3D scenes as structured object--relation--object graphs for spatial understanding. In observer-centric spatial perception, the same scene may be expressed under different local observer frames while its structure remains unchanged. However, existing models typically assume a fixed scene-aligned reference frame and may produce semantically inconsistent predictions when the scene is re-expressed in another observer frame. We attribute this failure to the heterogeneous frame dependency of relational predicates. Directional predicates such as left\textit{left}, front\textit{front}, right\textit{right}, and behind\textit{behind} are Observer-Dependent Relations\textbf{Observer-Dependent Relations}, whereas most contact, support, and semantic predicates, such as standing on\textit{standing on} and attached to\textit{attached to}, are approximately Observer-Independent Relations\textbf{Observer-Independent Relations}. Conventional models do not distinguish these frame responses, leading to degraded relation prediction under observer-frame reorientation. We introduce Observer-Aware Relations (OAR)\textbf{Observer-Aware Relations (OAR)}, which combines observer-aware geometric encoding and relation specialization, supported by frame-stable object encoding, for unified multi-label predicate prediction. Experiments on 3DSSG show that OAR consistently outperforms baselines across controlled observer-frame reorientations without training-time frame-reorientation augmentation, while remaining competitive on the standard benchmark. The project page is available at https://oar-predicate.github.io/.

PDF ↗

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace