Options
EdgeBoost: confidence boosting for resource constrained inference via selective offloading
Publikationstyp
Conference Paper
Date Issued
2024-04
Sprache
English
Author(s)
Christian-Albrechts-Universität zu Kiel
Start Page
11
End Page
18
Citation
20th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT) 2024
Publisher DOI
Scopus ID
Publisher
IEEE
ISBN
979-8-3503-6944-1
979-8-3503-6945-8
This paper introduces EdgeBoost, a selective input offloading system designed to overcome the challenges of limited computational resources on edge devices. EdgeBoost trains and calibrates a lightweight model for deployment on the edge and, in addition, deploys a large, complex model on the cloud. During inference, the edge model makes initial predictions for input samples, and if the confidence of the prediction is low, the sample is sent to the cloud model for further processing otherwise, we accept the local prediction. Due to careful calibration, EdgeBoost reduces the communication cost by 55%, 29% and 20% for CIFAR-100, ImageNet and Stanford Cars datasets, respectively, when compared to an cloud-only solution while achieving on par classification accuracy. Finally, EdgeBoost also achieves comparable accuracy to the state-of-the-art routing-based methods without the need for hosting the router on the edge.
Subjects
Inference Offloading | Lightweight Models | MCU | Model Calibration | Temperature Scaling | TinyML
MLE@TUHH
DDC Class
600: Technology