International Public Health Commons^™

Case Studies In Evaluating Time Series Prediction Models Using The Relative Mean Absolute Error, Nicholas G. Reich, Justin Lessler, Krzysztof Sakrejda, Stephen A. Lauer, Sopon Iamsirithaworn, Derek A T Cummings

Nicholas G Reich

Statistical prediction models inform decision-making processes in many real-world settings. Prior to using predictions in practice, one must rigorously test and validate candidate models to ensure that the proposed predictions have sufficient accuracy to be used in practice. In this paper, we present a framework for evaluating time series predictions that emphasizes computational simplicity and an intuitive interpretation using the relative mean absolute error metric. For a single time series, this metric enables comparisons of candidate model predictions against naive reference models, a method that can provide useful and standardized performance benchmarks. Additionally, in applications with multiple time series, this …