Engineering Commons^™

Metamorphic Robustness Testing: Exposing Hidden Defects In Citation Statistics And Journal Impact Factors, Zhi Q. Zhou, T.H. Tse, Matt Witheridge

Faculty of Engineering and Information Sciences - Papers: Part B

We propose a robustness testing approach for software systems that process large amounts of data. Our method usesmetamorphic relations to check software output for erroneous input in the absence of a tangible test oracle. We use this technique to testtwo major citation database systems: Scopus and the Web of Science. We report a surprising finding that the inclusion of hyphensin paper titles impedes citation counts, and that this is a result of the lack of robustness of the citation database systems in handlinghyphenated paper titles. Our results are valid for the entire literature as well as for individual fields such …