Filters

20252

Settings

Now showing1 - 2 of 2
  • Some of the metrics are blocked by your 
    consent settings
    Item type:Publication, Access status: without files ,
    Back to the roots: assessing mining techniques for Java vulnerability-contributing commits
    (Association for Computing Machinery (ACM), 2025-09-24)
    Hinrichs, Torge
    ;
    Iannone, Emanuele
    ;
    Tamás, Aladics
    ;
    Péter Hegedűs
    ;
    De Lucia, Andrea
    Context: Vulnerability-contributing commits (VCCs) are code changes that introduce vulnerabilities. Mining historical VCCs relies on SZZ-based algorithms that trace from known vulnerability-fixing commits. Objective: Although these techniques have been used, e.g., to train just-in-time vulnerability predictors, they lack systematic benchmarking to evaluate their precision, recall, and error sources. Method: We empirically assessed 12 VCC mining techniques in Java repositories using two benchmark datasets (one from the literature and one newly curated). We also explored combinations of techniques, through intersections, voting schemes, and machine learning, to improve performance. Results: Individual techniques achieved at most 0.60 precision but up to 0.89 recall. The precision rose to 0.75 when the outputs were combined with the logical AND, at the expense of recall. Machine learning ensembles reached 0.80 precision with a better precision–recall balance. Performance varied significantly by dataset. Analyzing “fixing commits” showed that certain fix types (e.g., filtering or sanitization) affect retrieval accuracy, and failure patterns highlighted weaknesses when fixes involve external data handling. Conclusion: Such results help software security researchers select the most suitable mining technique for their studies and understand new ways to design more accurate solutions.
    Publicationtype: Journal Article
      30
  • Some of the metrics are blocked by your 
    consent settings
    Item type:Publication, Access status: without files ,
    The ground truth effect: investigating SZZ variants in Just-in-Time vulnerability prediction
    (Springer, 2025)
    Cannavale, Alfonso
    ;
    Iannone, Emanuele
    ;
    Di Lillo, Gianluca
    ;
    Palomba, Fabio
    ;
    De Lucia, Andrea
    Just-in-Time (JIT) vulnerability prediction is critical for proactively securing software, yet its effectiveness heavily relies on the quality of the ground truth used for training models. This ground truth is commonly established using variants of the SZZ algorithm to identify vulnerability-contributing commits (VCCs). However, the impact of choosing a specific SZZ variant on model performance remains largely unexplored. In this study, we systematically investigate the effect of eight SZZ variants on JIT vulnerability prediction across seven open-source Java projects. Our findings reveal that the choice of the SZZ variant is a non-trivial factor. Models trained with datasets labeled by variants like B-SZZ, V-SZZ, and VCC-SZZ achieve strong and stable predictive performance, with median MCC scores often exceeding 0.50. In contrast, variants such as L-SZZ and R-SZZ produce models that perform no better than random chance, with median MCC scores close to 0.0. This performance gap demonstrates that an inappropriate SZZ variant can invalidate prediction models, underscoring the necessity of a principled approach to defining ground truth.
    Publicationtype: Conference Paper
      32