Computer Sciences Commons^™

Code Execution Capability As A Metric For Machine Learning–Assisted Software Vulnerability Detection Models, Daniel Grahn, Lingwei Chen, Junjie Zhang

Computer Science and Engineering Faculty Publications

In this paper, we consider how the ability to learn Code Execution Tasks affects a model’s accuracy on software vulnerability detection (SVD) benchmark datasets. We initially find that models can achieve near state-of-the-art accuracy on SVD benchmarks regardless of their ability to learn Code Execution Tasks. However, these models fail to generalize well across SVD benchmarks. The results indicate a bias in the datasets that allows models to predict non- SVD signals. Under the theory that different collection methods will reduce biases, we investigate combining the SVD datasets. When trained on combined datasets, SVD accuracy is reduced but correlation with …