109 results
Now showing1 - 10 of 109
- Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Emerging challenges in threat modeling for GenAI-augmented systems: a view from the trenches(2026-10); ; ; 15 2 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Automatic rule checking for microservices: supporting security analysis with explainability(Association for Computing Machinery (ACM), 2026-07-16); ; ; ; Software security analysis is often done manually, raising performance and correctness issues. Introducing automation is challenging because human verification of the outcomes is often required, especially for security assessment and certification. The distributed nature of microservice applications further increases these concerns. We present an approach for automatically checking architectural security rules on models of microservice applications. It provides explainability for verdicts of rules that are expressed as model queries in our rule specification language. This comprehensible, step-by-step evidence leverages traceability information from the input models to link to artifacts in code. Hence, the complete analysis process from source code via model to rule verdict can be traced and verified. Custom rules can be formulated in addition to a library of 25 best-practice architectural security rules. We evaluated the approach’s correctness by checking the 25 rules on 16 dataflow diagrams of microservice applications with a prototype (called MicroCertiSec) and observed promising results (precision=0.98; recall=1). Additionally, we performed an evaluation with industry experts and academics to gain initial insights into the approach’s usefulness for real-world security analysis. The nine participants gave highly positive feedback on usefulness and usability and stated they would use such an approach in their daily work.Publicationtype: Journal Article35 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , A match made in heaven? AI-driven matching of vulnerabilities and security unit tests(Association for Computing Machinery (ACM), 2026-04); ; Software vulnerabilities are often detected via taint analysis, penetration testing, or fuzzing. They are also found via unit tests that exercise security-sensitive behavior with specific inputs, called vulnerability-witnessing tests. Generative AI models could help developers in writing them, but they require many examples to learn from, which are currently scarce. This paper introduces VuTeCo, an AI-driven framework for collecting examples of vulnerability-witnessing tests from Java repositories. VuTeCo carries out two tasks: (1) The "Finding"task to determine whether a unit test case is security-related, and (2) the "Matching"task to relate a test case to the vulnerability it witnesses. VuTeCo addresses the Finding task with UniXcoder, achieving an F₀.₅ score of 0.73 and a precision of 0.83 on a test set of unit tests from Vul4J. The Matching task is addressed using DeepSeek Coder, achieving an F₀.₅ score of 0.65 and a precision of 0.75 on a test set of pairs of unit tests and vulnerabilities from Vul4J. VuTeCo has been used in the wild on 427 Java projects and 1,238 vulnerabilities, obtaining 224 test cases confirmed to be security-related and 35 tests correctly matched to 29 vulnerabilities. The validated tests were collected in a new dataset called Test4Vul. VuTeCo lays the foundation for large-scale retrieval of vulnerability-witnessing tests, enabling future AI models to better understand and generate security unit tests.Publicationtype: Conference Paper19 7 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , VULTERMINATOR: bringing back template-based automated repair for fixing Java vulnerabilitiesMainstream techniques for Automated Vulnerability Repair (AVR) lean heavily on Large Language Models (LLMs) and treat the vulnerability repair as a code translation task. Yet, their effectiveness is limited due to the complex nature of vulnerability fixes and, possibly, the lack of training datasets in the Java programming language. On the other hand, templatebased Automated Program Repair (APR) remains a popular way to fix general bugs. However, only a few approaches have ever employed vulnerability-specific fix templates. This paper introduces VulTerminator, a novel repair approach for Java vulnerabilities that leverages both heuristic-based and data-driven fix templates. The former are specialized for certain vulnerability types, such as XML External Entity (XXE) injection, which can be more easily patched with predefined heuristics. The latter aim to repair a broader class of vulnerabilities by generating common patch templates with masks, which are later filled by a fine-tuned Masked Language Model (MLM). In this paper, we introduce a total of eleven fix templates distilled from real-world Java patches and evaluate VulTerminator on 106 vulnerabilities with test cases from Vul4J+, as well as on 169 unseen vulnerabilities from a newly curated dataset called Vul4JL. VulTerminator achieves the best overall repair performance, outperforming the state-of-the-art approaches by 7 % on Vul4J+ and 27 % on Vul4JL, as confirmed by manual inspection. Vulterminator managed to fix 10 vulnerabilities in Vul4J+ and 16 in Vul4JL that no other approach could do, mainly due to the contribution of heuristicbased templates.Publicationtype: Conference Paper15 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Discrete prompt optimization using genetic algorithm for secure Python code generationLarge language models (LLMs) have become powerful tools that enable novice developers to generate production-level code. However, research has highlighted the security risks associated with such code generation, due to the high volume of generated software vulnerabilities. Recent studies have explored various techniques for automatically optimizing prompts to elicit desired responses from LLMs. Among these methods, Genetic Algorithms (GAs), which search for optimal solutions by evolving an initial population of candidates through iterative mutations, have gained attention as a lightweight and effective prompt optimization approach that does not require large datasets or access to model weights. However, their potential has not yet been examined in the context of secure code generation. In this paper, we use GA to develop a discrete prompt optimization pipeline specifically designed for secure code generation. We introduce two domain-specific prompt mutation techniques and assess how incorporating these security-focused mutations alongside general-purpose techniques, such as back translation and paraphrasing, affects the security of Python code generated by LLMs. Results demonstrate that our security-specific mutation techniques led to prompts with richer security context compared to the generic mutation techniques. Furthermore, combining these techniques with generic mutations substantially reduced the number of security weaknesses in the LLM-generated code. We also observed that prompts optimized for a particular LLM tend to perform best on that same model, highlighting the importance of model-specific prompt optimization.Publicationtype: Journal Article37 19 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Beyond prompting: the role of phrasing tasks in vulnerability prediction for JavaPredicting vulnerability in a code element, such as a function or method, often leverages machine or deep learning models to classify whether it is vulnerable or not. Recently, novel solutions exploiting conversational large language models (LLMs) have emerged, which allow the formulation of the task through a prompt containing natural language elements and the input code element, obtaining a natural language as a response. Although initial promising results, there is currently no broad exploration of (i) how the input prompt influences the prediction capabilities and (ii) what characteristics of the model response relate to correct predictions. In this paper, we conduct an empirical investigation into how accurately two popular conversational LLMs, i.e., GPT-3.5 and Llama-2, predict whether a Java method is vulnerable by employing a thorough prompting strategy by (i) adhering to the Zero-Shot and Zero-Shot Chain-of-Thought techniques and (ii) formulating the prediction task in alternative ways via rephrasing. After a manual inspection of the responses generated, we observed that GPT-3.5 displayed more variable F1 scores compared to Llama-2, which was steadier but often gave no direct classification. ZS prompts achieved F1 scores between 0.53 and 0.69, with a tendency of classifying methods positively (i.e., ‘vulnerable’); conversely, ZS-CoT presents a broader range of scores, varying from 0.35 to 0.72, with often inconsistencies in the results. Then, we phrased the task in their “inverted form”, i.e., asking the LLM to check for the absence of vulnerabilities, which led to worse results for GPT-3.5, while Llama-2 occasionally performed better. The study further suggests that textual metrics provide important information on LLM outputs. Despite this, these metrics are not correlated with actual outcomes, as the models respond consistently with uniform confidence, irrespective of whether the outcome is correct or not. This underscores the need for customized prompt engineering and response analysis strategies to improve the precision and reliability of LLM-based systems for vulnerability prediction. In addition, we applied our study to two state-of-the-art LLMs, validating the broader applicability of our methodology. Finally, we performed an analysis of various textual properties of the model responses, such as response length and readability scores, to further explore the characteristics of the responses given for vulnerability detection tasks.Publicationtype: Journal Article17 20 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Prompting techniques for secure code generation: a systematic investigation(Association for Computing Machinery (ACM), 2025-10-04); ; ; ; Large Language Models (LLMs) are gaining momentum in software development with prompt-driven programming enabling developers to create code from Natural Language (NL) instructions. However, studies have questioned their ability to produce secure code and, thereby, the quality of prompt-generated software. Alongside, various prompting techniques that carefully tailor prompts have emerged to elicit optimal responses from LLMs. Still, the interplay between such prompting strategies and secure code generation remains underexplored and calls for further investigations. Objective: In this study, we investigate the impact of different prompting techniques on the security of code generated from NL instructions by LLMs. Method: First, we perform a systematic literature review to identify the existing prompting techniques that can be used for code generation tasks. A subset of these techniques are evaluated on GPT-3, GPT-3.5, and GPT-4 models for secure code generation. For this, we used an existing dataset consisting of 150 NL security-relevant code generation prompts. Results: Our work (i) classifies potential prompting techniques for code generation (ii) adapts and evaluates a subset of the identified techniques for secure code generation tasks, and (iii) observes a reduction in security weaknesses across the tested LLMs, especially after using an existing technique called Recursive Criticism and Improvement (RCI), contributing valuable insights to the ongoing discourse on LLM-generated code security.Publicationtype: Journal Article40 50 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Comparison of static analysis architecture recovery tools for microservice applications(Springer, 2025-10-01); ; ; ; Architecture recovery tools help software engineers obtain an overview of the structure of their software systems during all phases of the software development life cycle. This is especially important for microservice applications because they consist of multiple interacting microservices, which makes it more challenging to oversee the architecture. Various tools and techniques for architecture recovery (also called architecture reconstruction) have been presented in academic and gray literature sources, but no overview and comparison of their accuracy exists. This paper presents the results of a multivocal literature review with the goal of identifying architecture recovery tools for microservice applications and a comparison of the identified tools’ architectural recovery accuracy. We focused on static tools since they can be integrated into fast-paced CI/CD pipelines. 13 such tools were identified from the literature and nine of them could be executed and compared on their capability of detecting different system characteristics. The best-performing tool exhibited an overall F1-score of 0.86. Additionally, the possibility of combining multiple tools to increase the recovery correctness was investigated, yielding a combination of four individual tools that achieves an F1-score of 0.91.Publicationtype: Journal Article17 22 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Back to the roots: assessing mining techniques for Java vulnerability-contributing commits(Association for Computing Machinery (ACM), 2025-09-24); ; ; 30 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Retrieve, Refine, or Both? Using Task-Specific Guidelines for Secure Python Code GenerationLarge Language Models (LLMs) are increasingly used for code generation, but they often produce code with security vulnerabilities. While techniques like fine-tuning and instruction tuning can improve security, they are computationally expensive and require large amounts of secure code data. Recent studies have explored prompting techniques to enhance code security without additional training. Among these, Recursive Criticism and Improvement (RCI) has demonstrated strong improvements by iteratively refining the generated code by leveraging LLMs' self-critiquing capabilities. However, RCI relies on the model's ability to identify security flaws, which is constrained by its training data and susceptibility to hallucinations. This paper investigates the impact of incorporating taskspecific secure coding guidelines extracted from MITRE's CWE and CodeQL recommendations into LLM prompts. For this, we employ Retrieval-Augmented Generation (RAG) to dynamically retrieve the relevant guidelines that help the LLM avoid generating insecure code. We compare RAG with RCI, observing that both deliver comparable performance in terms of code security, with RAG consuming considerably less time and fewer tokens. Additionally, combining both approaches further reduces the amount of insecure code generated, requiring only slightly more resources than RCI alone, highlighting the benefit of adding relevant guidelines in improving LLM-generated code security.Publicationtype: Conference Paper35