Investigation of continous human action recognition for manual assembly using the HARDAT dataset

Human Action Recognition (HAR) enables automated systems to interpret human behavior, making it a valuable feedback mechanism for assistance systems in manual assembly. This paper investigates the application of state of the art HAR algorithms, originally developed in domains such as sports, surveillance, and daily activities, to the context of industrial manual assembly. Using the HARDAT dataset, which provides multimodal sensor data from typical assembly tasks, several advanced architectures were evaluated. The focus was placed on four leading models, including graph based and transformer based approaches that process skeletal and RGB input data. Additional established graph based models were included to provide a more comprehensive performance assessment. All models were assessed under continuous recognition conditions using a sliding window approach. The results show that graph based models, particularly ProtoGCN and CTR-GCN, achieved the highest performance in terms of F1 score and accuracy. The findings emphasize the importance of appropriate temporal segmentation and data quality when applying HAR in real world industrial settings. This work also provides reproducible training pipelines to support further research and implementation in assembly assistance systems.

Subjects

Artificial Intelligence

Assembly

Assistance System

Continuous Action Recognition

Human Action Recognition

Methods-Time-Measurement

DDC Class

006.3: Artificial Intelligence

620: Engineering

Lizenz

https://creativecommons.org/licenses/by-nc-nd/4.0/

Publication version

publishedVersion

Name

1-s2.0-S2212827126011856-main.pdf

Type

Main Article

Size

449.68 KB

Format

Adobe PDF

Options

Investigation of continous human action recognition for manual assembly using the HARDAT dataset