Profentzas, ChristosChristosProfentzasAlmgren, MagnusMagnusAlmgrenLandsiedel, OlafOlafLandsiedel2025-02-062025-02-062021-05Proceedings of the Workshop on Benchmarking Cyber-Physical Systems and Internet of Things: 32-37 (2021)978-1-4503-8439-1https://hdl.handle.net/11420/53872Advances in deep learning have revolutionized machine learning by solving complex tasks such as image, speech, and text recognition. However, training and inference of deep neural networks are resource-intensive. Recently, researchers made efforts to bring inference to IoT edge and sensor devices which have become the prime data sources nowadays. However, running deep neural networks on low-power IoT devices is challenging due to their resource-constraints in memory, compute power, and energy. This paper presents a benchmark to grasp these trade-offs by evaluating three representative deep learning frameworks: uTensor, TF-Lite-Micro, and CMSIS-NN. Our benchmark reveals significant differences and trade-offs for each framework and its tool-chain: (1) We find that uTensor is the most straightforward framework to use, followed by TF-Micro, and then CMSIS-NN. (2) Our evaluation shows large differences in energy, RAM, and Flash footprints. For example, in terms of energy, CMSIS-NN is the most efficient, followed by TF-Micro and then uTensor, each with a significant gap.endeep neural networks | IoT | low-powerMLE@TUHHTechnology::600: TechnologyConference Paper10.1145/3458473.3458823Conference Paper