Physical Sciences and Mathematics Commons^™

Evaluating Machine Learning Classifiers For Defensive Cyber Operations, Michael D. Rich, Robert F. Mills, Thomas E. Dube, Steven K. Rogers

Military Cyber Affairs

Today’s defensive cyber sensors are dominated by signature-based analytical methods that require continuous maintenance and lack the ability to detect unknown threats. Anomaly detection offers the ability to detect unknown threats, but despite over 15 years of active research, the operationalization of anomaly detection and machine learning for Defensive Cyber Operations (DCO) is lagging. This article provides an introduction to machine learning concepts with a focus on the unique challenges to using machine learning for DCO. Traditional machine learning evaluation methods are challenged in favor of a value-focused evaluation method that incorporates evaluator-specific weights for classifier and sensitivity threshold selection …