Using 2D scene graphs as an enabler for DT topology

Scan-to-BIM research has gained significant attention, yet many reconstruction methodologies overlook the crucial topological relationships in Building Information Modeling (BIM). To address this challenge, we propose using Scene Graphs to capture contextual relationships in images, focusing on hallways and offices at the Technical University of Munich (TUM). Our approach involves fine-tuning an existing Scene Graph Generation (SGG) model and proposing an in-house model both aiming to predict scene graphs, by integrating object detection and predicate detection methods. The fine-tuned SGG model with the benchmark archived the recall@50 of 95.57 in predicate classification mode. Comparatively, our in-house model attained a recall of 65.12 for the overall scene graph generation. Despite these promising results, some limitations remain, such as low object detection accuracy and the exclusion of non-relationships, as well as the evaluation being limited to qualitative comparisons and was done independently for each method. These limitations highlight areas for future work. Nevertheless, this study offers a proof of concept for integrating scene graph predictions into Scan-to-BIM workflows while identifying areas for further improvement.

Subjects

Closed-Vocabulary 2D Scene-Graph

Computer Vision

Digital Twin

Rule-Based ML

Transfer Learning

DDC Class

005: Computer Programming, Programs, Data and Security

006: Special computer methods

720: Architecture

Lizenz

https://creativecommons.org/licenses/by/4.0/

Name

Using 2D scene graphs as an enabler for DT topology.pdf

Size

916.77 KB

Format

Adobe PDF

Options

Using 2D scene graphs as an enabler for DT topology