Schyga, JakobJakobSchygaHinckeldeyn, JohannesJohannesHinckeldeynKreutzfeldt, JochenJochenKreutzfeldt2022-04-262022-04-262022-04-06Sensors 22 (7): 2797 (2022)http://hdl.handle.net/11420/12380Despite their enormous potential, the use of indoor localization systems (ILS) remains seldom. One reason is the lack of market transparency and stakeholders’ trust in the systems’ performance as a consequence of insufficient use of test and evaluation (T&E) methodologies. The heterogeneous nature of ILS, their influences, and their applications pose various challenges for the design of a methodology that provides meaningful results. Methodologies for building-wide testing exist, but their use is mostly limited to associated indoor localization competitions. In this work, the <i>T&E 4iLoc Framework</i> is proposed—a methodology for T&E of indoor localization systems in semi-controlled environments based on a system-level and black-box approach. In contrast to building-wide testing, T&E in semi-controlled environments, such as test halls, is characterized by lower costs, higher reproducibility, and better comparability of the results. The limitation of low transferability to real-world applications is addressed by an application-driven design approach. The empirical validation of the <i>T&E 4iLoc Framework</i>, based on the examination of a contour-based light detection and ranging (LiDAR) ILS, an ultra wideband ILS, and a camera-based ILS for the application of automated guided vehicles in warehouse operation, demonstrates the benefits of T&E with the <i>T&E 4iLoc Framework</i>.Despite their enormous potential, the use of indoor localization systems (ILS) remains seldom. One reason is the lack of market transparency and stakeholders’ trust in the systems’ performance as a consequence of insufficient use of test and evaluation (T&E) methodologies. The heterogeneous nature of ILS, their influences, and their applications pose various challenges for the design of a methodology that provides meaningful results. Methodologies for building-wide testing exist, but their use is mostly limited to associated indoor localization competitions. In this work, the <i>T&E 4iLoc Framework</i> is proposed—a methodology for T&E of indoor localization systems in semi-controlled environments based on a system-level and black-box approach. In contrast to building-wide testing, T&E in semi-controlled environments, such as test halls, is characterized by lower costs, higher reproducibility, and better comparability of the results. The limitation of low transferability to real-world applications is addressed by an application-driven design approach. The empirical validation of the <i>T&E 4iLoc Framework</i>, based on the examination of a contour-based light detection and ranging (LiDAR) ILS, an ultra wideband ILS, and a camera-based ILS for the application of automated guided vehicles in warehouse operation, demonstrates the benefits of T&E with the <i>T&E 4iLoc Framework</i>.en1424-8220Sensors20227Multidisciplinary Digital Publishing Institutehttps://creativecommons.org/licenses/by/4.0/indoor localizationtest and evaluationmethodologybenchmarkingTechnikMeaningful test and evaluation of indoor localization systems in semi-controlled environmentsJournal Article2022-04-1110.15480/882.431210.3390/s2207279710.15480/882.4312Journal Article