The effect of annotation quality on wear semantic segmentation by CNN

In this work, we investigate the impact of annotation quality and domain expertise on the performance of Convolutional Neural Networks (CNNs) for semantic segmentation of wear on titanium nitride (TiN) and titanium carbonitride (TiCN) coated end mills. Using an innovative measurement system and customized CNN architecture, we found that domain expertise significantly affects model performance. Annotator 1 achieved maximum mIoU scores of 0.8153 for abnormal wear and 0.7120 for normal wear on TiN datasets, whereas Annotator 3 with the lowest expertise achieved significantly lower scores. Sensitivity to annotation inconsistencies and model hyperparameters were examined, revealing that models for TiCN datasets showed a higher coefficient of variation (CV) of 16.32% compared to 8.6% for TiN due to the subtle wear characteristics, highlighting the need for optimized annotation policies and high-quality images to improve wear segmentation.

Subjects

annotation protocols

annotation quality

domain expertise

image annotation

IoU metrics

labeling quality

neural network performance

semantic segmentation

U-Net model

wear detection| machining tools

DDC Class

621: Applied Physics

006.3: Artificial Intelligence

620.11: Engineering Materials

621.38: Electronics, Communications Engineering

Lizenz

https://creativecommons.org/licenses/by/4.0/

Name

sensors-24-04777-v2.pdf

Size

26.65 MB

Format

Adobe PDF

Options

The effect of annotation quality on wear semantic segmentation by CNN