A portable compiler-runtime approach for scalability prediction

Highly scalable parallel applications can efficiently solve expensive computational problems when run on a large number of compute nodes. However, selecting the optimal number of nodes for a compute job of a given size is non-trivial, and allocating too few or too many nodes may not yield the expected performance. Knowing the scaling behavior of an application in advance enables us, for example, to make optimal use of the available hardware resources. We introduce a novel, portable approach to predict the scalability of parallel applications written in modern high-level programming models. We propose a predictive compiler-runtime framework based on Celerity, a task-based distributed runtime system that enables executing SYCL codes on clusters. The framework targets a broad range of computing systems, from CPU to GPU clusters, and proposes a model that combines machine learning, communication modeling and DAG heuristics. Experimental results on two large-scale clusters, JUWELS and Marconi-100, show accurate scalability prediction of unseen single and multi-task applications.

DDC Class

004: Computer Sciences

005: Computer Programming, Programs, Data and Security

Funding(s)

CELERITY

More Funding Information

This document is the results of the research project partially funded by FWF (I 3388) and DFG (CO 1544/1-1, project number 360291326) as part of the DACH project CELERITY.

Lizenz

https://creativecommons.org/licenses/by/4.0/

Publication version

publishedVersion

Name

1-s2.0-S0167739X25006314-main.pdf

Size

13.15 MB

Format

Adobe PDF

Options

A portable compiler-runtime approach for scalability prediction