A digital assistance system leveraging vision foundation models & 3D localization for reproducible defect segmentation in visual inspection

In response to the limitations of manual visual inspection in manufacturing industries, this paper presents a novel approach that combines Meta AI’s Segment Anything Model (SAM) as a current representative of a vision foundation model with marker-based localization techniques to enhance quality assurance processes. Focusing on aircraft cabin components, where product complexity and variance pose significant challenges, a digital assistance system is developed. By integrating SAM-based segmentation and defect localization on the CAD model, the proposed system facilitates significant parts of the inspection workflow, thereby increasing efficiency and reproducibility. The system also enables the collection of comprehensive data sets, which is essential for the refinement of AI models, e.g. in transfer learning approaches. A two-pronged evaluation approach includes the assessment of localization accuracies in an experimental setup as well as the evaluation of usability and workload in an industry-oriented user study. The novel segmentation approach using the SAM achieved high usability, providing an efficient and user-friendly solution for the application domain, while maintaining a reasonable workload for the operators, highlighting its significant potential for streamlining industrial inspection processes.

DDC Class

658.5: Of Production

Publication version

publishedVersion

Lizenz

https://creativecommons.org/licenses/by-nc-nd/4.0/

Name

1-s2.0-S2212827124012629-main.pdf

Type

Main Article

Size

2.12 MB

Format

Adobe PDF

Options

A digital assistance system leveraging vision foundation models & 3D localization for reproducible defect segmentation in visual inspection