Digital Commons Network^™

The university research environment has been undergoing profound change in recent decades. Aiming at international competitiveness and excellence, a variety of polices have been designed and implemented in many countries. However, evidence-based analysis of policy effects is scarce. This paper develops methods for evaluating the effect of university research policy on university system research input-output dynamics. We assume stable dynamics between inputs and outputs, and that effective policy change introduces external interventions and therefore structural changes into the system. Our proposed method involves three steps: modeling system dynamics, detecting structural change, and mapping policy change. Examining the case of the …

The university research environment has been undergoing profound change in recent decades and performance-based research funding systems (PRFSs) are one of the many novelties introduced. This paper seeks to find general lessons in the accumulated experience with PRFSs that can serve to enrich our understanding of how research policy and innovation systems are evolving. The paper also links the PRFS experience with the public management literature, particularly new public management, and understanding of public sector performance evaluation systems. PRFSs were found to be complex, dynamic systems, balancing peer review and metrics, accommodating differences between fields, and involving lengthy consultation with …

Creative use of bibliometric analysis in evaluation offers an unparalleled opportunity to take advantage of the rich information embedded in the written products of scientific work to track the output and influence of funded scholars. Many metrics and techniques have been developed: from publication and citation counts to percentile rankings, h-index, impact factor, maps of the knowledge landscape, maps of geographical distribution, and metrics of interdisciplinarity and specialization. Analysis can demonstrate evolution over long periods of time, and can draw quantitative comparisons among subgroups or with others anywhere in the world. It would be dangerous to consider such data and …

Since 1980, national university departmental ranking exercises have developed in several countries. This paper reviews exercises in the U.S., U.K. and Australia to assess the state-of-the-art and to identify common themes and trends. The findings are that the exercises are becoming more elaborate, even unwieldy, and that there is some retreat from complexity. There seems to be a movement towards bibliometric measures. The exercises also seem to be effective in enhancing university focus on research strategy.

Research evaluation in the USA historically tended to rely more heavily on peer review than on bibliometric method, but interest in quantitative methods including bibliometrics appears to be growing. In this paper, we discuss the use of bibliometric techniques of research evaluation by the US federal government over the past decade. Within the past decade, commentators have pointed to something of a rebirth of interest in evaluation along with pressure on agencies to develop quantitative indicators. Evaluation of economic and societal outcomes of research has become a priority. Bibliometric method continues to evolve in response to these needs and therefore …