Berger, Bernhard JohannesBernhard JohannesBergerPlump, ChristinaChristinaPlump2024-01-252024-01-252023-1026th International Conference on Model Driven Engineering Languages and Systems (MODELS 2023)https://hdl.handle.net/11420/45317Threat Modeling is an essential step in secure software system development. It is a manual, attacker-centric approach for identifying architecture-level security flaws during the planning phase of software systems. In the last years, academia presented two methods to automate threat detection that do not focus on a particular class of security flaws but offer general-purpose means to describe security flaws.This paper compares both approaches on an equal data foundation that was published with one of the approaches. Therefore, we specify a model-To-model transformation for converting between the approaches to allow this conceptual replication. Additionally, we provide security flaw patterns for the second approach that any user of the approach can use. We then replicate the detection with the second security flaw detection approach to compare both approaches. We focus our analysis on differences between automation-specific and approach-specific finding misclassifications on identifying whether some flaws are harder to find with an automated approach than others.We find that missed flaws usually stem from the imprecise definition of security flaws, while incorrectly identified flaws are approach-dependent. Despite that, both approaches perform similarly. The knowledge base, the transformation scripts and the evaluation script are publicly available to support the research community.enautomationcomparisondataflow diagramsinteroperabilitysecurity flaw detectionthreat modelingEngineering and Applied OperationsConference Paper10.1109/MODELS58315.2023.00027Conference Paper