Options
Projekt Titel
Neural ODE training via stochastic control and uncertainty quantification
Förderkennzeichen
FA 1268/11-1
Funding code
945.03-1097
Startdatum
January 1, 2026
Enddatum
December 31, 2029
Gepris ID
Loading...
This project is part of the research unit "Active Learning for Systems and Control (ALeSCo) - Data Informativity, Uncertainty, and Guarantees". Data-driven and learning-based approaches for modelling of dynamic systems and for design of control laws have gained prominence in recent years. Due to their universal approximation properties, neural networks in different variants and architectures are among the most frequently considered learning methods in systems and control. In contrast to this trend, this project does not ask what machine learning can do for control. Rather we explore the question of how systems and control methods, in particular concepts from uncertainty quantification and stochastic control, can be beneficial in the design and analysis of training formulations for neural networks. Specifically, the project considers Neural Ordinary Differential Equations (NODEs) and their explicit discretizations which take the form of Residual Networks (ResNets). We formulate and analyze the propagation of data through NODEs and ResNets in the framework of Polynomial Chaos Expansions (PCE) of L2 random variables, a concept proposed by Norbert Wiener. These random variables are used to model the input data of neural networks, which enables the forward propagation of entire sets of input data. Using the PCE description of the propagated data in the output layer, we explore the analysis of generalization properties. We extend the uncertainty propagation towards the training of NODEs and ResNets combining the PCE-based L2 framework with optimal control formulations of the training problem. We explore how generalization properties of neural networks can be directly considered in the training problems and how system-theoretic dissipativity notions of optimal control problems allow for performance-preserving pruning of trained networks. Moreover, we devise active learning strategies based on the conceived L2 training problems. Specifically, we investigate how the propagated input data enables uncertainty quantification for label predictions. To this end, we investigate novel data informativity notions tailored to neural networks. Finally, we explore how stochastic control concepts, i.e. feedback policies, can be leveraged to design neural networks with quantifiable generalization properties. The investigated methods are evaluated on benchmark problems stemming from the machine learning literature and on systems and control specific benchmarks developed in the research unit ALeSCo.