Options
Critical evaluation of LOCO dataset with machine learning
Citation Link: https://doi.org/10.15480/882.4692
Publikationstyp
Conference Paper
Date Issued
2022-09
Sprache
English
Author(s)
Herausgeber*innen
Institut
TORE-DOI
First published in
Number in series
33
Start Page
177
End Page
206
Citation
Hamburg International Conference of Logistics (HICL) 33: 177-206 (2022)
Contribution to Conference
Scopus ID
Publisher
epubli
Peer Reviewed
true
Purpose: Object detection is rapidly evolving through machine learning technology in automation systems. Well prepared data is necessary to train the algorithms. Accordingly, the objective of this paper is to describe a re-evaluation of the so-called Logistics Objects in Context (LOCO) dataset, which is the first dataset for object detection in the field of intralogistics.
Methodology: We use an experimental research approach with three steps to evaluate the LOCO dataset. Firstly, the images on GitHub were analyzed to understand the dataset better. Secondly, Google Drive Cloud was used for training purposes to revisit the algorithmic implementation and training. Lastly, the LOCO dataset was examined, if it is possible to achieve the same training results in comparison to the original publications.
Findings: The mean average precision, a common benchmark in object detection, achieved in our study was 64.54%, and shows a significant increase from the initial study of the LOCO authors, achieving 41%. However, improvement potential is seen specifically within object types of forklifts and pallet truck.
Originality: This paper presents the first critical replication study of the LOCO dataset for object detection in intralogistics. It shows that the training with better hyperparameters based on LOCO can even achieve a higher accuracy than presented in the original publication. However, there is also further room for improving the LOCO dataset.
Methodology: We use an experimental research approach with three steps to evaluate the LOCO dataset. Firstly, the images on GitHub were analyzed to understand the dataset better. Secondly, Google Drive Cloud was used for training purposes to revisit the algorithmic implementation and training. Lastly, the LOCO dataset was examined, if it is possible to achieve the same training results in comparison to the original publications.
Findings: The mean average precision, a common benchmark in object detection, achieved in our study was 64.54%, and shows a significant increase from the initial study of the LOCO authors, achieving 41%. However, improvement potential is seen specifically within object types of forklifts and pallet truck.
Originality: This paper presents the first critical replication study of the LOCO dataset for object detection in intralogistics. It shows that the training with better hyperparameters based on LOCO can even achieve a higher accuracy than presented in the original publication. However, there is also further room for improving the LOCO dataset.
Subjects
Artificial Intelligence
Blockchain
DDC Class
004: Informatik
Publication version
publishedVersion
Loading...
Name
Savas and Hinckeldeyn (2022) - Critical Evaluation of LOCO dataset with Machine Learning.pdf
Size
1.41 MB
Format
Adobe PDF