Synthesizing adaptive test strategies from temporal logic specifications

Constructing good test cases is difficult and time-consuming, especially if the system under test is still under development and its exact behavior is not yet fixed. We propose a new approach to compute test strategies for reactive systems from a given temporal logic specification using formal methods. The computed strategies are guaranteed to reveal certain simple faults in every realization of the specification and for every behavior of the uncontrollable part of the system’s environment. The proposed approach supports different assumptions on occurrences of faults (ranging from a single transient fault to a persistent fault) and by default aims at unveiling the weakest one. We argue that such tests are also sensitive for more complex bugs. Since the specification may not define the system behavior completely, we use reactive synthesis algorithms with partial information. The computed strategies are adaptive test strategies that react to behavior at runtime. We work out the underlying theory of adaptive test strategy synthesis and present experiments for a safety-critical component of a real-world satellite system. We demonstrate that our approach can be applied to industrial specifications and that the synthesized test strategies are capable of detecting bugs that are hard to detect with random testing.

Subjects

automatic test case generation

system testing

specification testing

adaptive tests

synthesis

reactive systems

mutation testing

DDC Class

620: Ingenieurwissenschaften

Lizenz

https://creativecommons.org/licenses/by/4.0/

Name

Bloem2019_Article_SynthesizingAdaptiveTestStrate.pdf

Size

838.75 KB

Format

Adobe PDF

Options

Synthesizing adaptive test strategies from temporal logic specifications