Grand Challenges In Us Science Policy Attempt Policy Innovation, Diana Hicks
Dec 2015
Grand Challenges In Us Science Policy Attempt Policy Innovation, Diana Hicks
Diana Hicks
This paper investigates the historical development of the Grand Challenges concept in US science policy. The concept originated in advocacy for funding for high performance computing and was enshrined in the High Performance Computing Act of 1991. The Bill & Melinda Gates Foundation’s Grand Challenges in Global Health program marked a second milestone in the application of the concept to US science funding. The National Academy of Engineering’s Grand Challenges in Engineering followed in 2008. Most recently the White House has pursued programs under the Grand Challenges rubric. The history of these varied initiatives spanning 40 years is examined here …
Narrative Visualization Of The Outcomes Of Federal Investments In Research, Diana Hicks
Dec 2015
Narrative Visualization Of The Outcomes Of Federal Investments In Research, Diana Hicks
Diana Hicks
We offer here a narrative visualization entitled: Technology hot spots and the Office of Science and position its contribution within discussion of novel forms of communicating research results as an aid to maximizing use of research evaluation. In this study, patent co-citation analysis was used to systematically identify emerging high impact technologies in the US technology ecosystem and then to establish that Office of Science of the US Department of Energy (DOE) funds research that underpins these technology “hot spots.” We discuss the strengths and weaknesses of this novel form of communicating evaluation results based our experience. The video at …
The New York Times As A Resource For Mode 2, Diana Hicks, Jian Wang
Oct 2013
The New York Times As A Resource For Mode 2, Diana Hicks, Jian Wang
Diana Hicks
The New York Times receives more citations from academic journals than the American Sociological Review, Research Policy, or the Harvard Law Review. This paper explores the reasons why scholars cite the New York Times so much. Reasons include studying the newspaper itself or New York City, establishing public interest in a topic by referencing press coverage, introducing specificity, and treating the New York Times very much like an academic journal. The phenomenon seems to reflect a Mode 2 type of scholarship produced in the context of application, organizationally diverse, socially accountable and aiming to be socially useful as well as …
Detecting Structural Change In University Research Systems: A Case Study Of British Research Policy, Jian Wang, Diana Hicks
Jul 2013
Detecting Structural Change In University Research Systems: A Case Study Of British Research Policy, Jian Wang, Diana Hicks
Diana Hicks
The university research environment has been undergoing profound change in recent decades. Aiming at international competitiveness and excellence, a variety of polices have been designed and implemented in many countries. However, evidence-based analysis of policy effects is scarce. This paper develops methods for evaluating the effect of university research policy on university system research input-output dynamics. We assume stable dynamics between inputs and outputs, and that effective policy change introduces external interventions and therefore structural changes into the system. Our proposed method involves three steps: modeling system dynamics, detecting structural change, and mapping policy change. Examining the case of the …
Pathways From Discovery To Commercialization: Using Web Sources To Track Small And Medium-Sized Enterprise Strategies In Emerging Nanotechnologies, Jan Youtie, Diana Hicks, Philip Shapira, Travis Horsely
Oct 2012
Pathways From Discovery To Commercialization: Using Web Sources To Track Small And Medium-Sized Enterprise Strategies In Emerging Nanotechnologies, Jan Youtie, Diana Hicks, Philip Shapira, Travis Horsely
Diana Hicks
There is a growing need for fresh and systematic evidence about company innovation in emerging technologies such as nanotechnology. It is particularly important to track shifts from discovery to commercialization at the later stages of the innovation process, where diverse product and financial strategies may be pursued. This paper presents results from a pilot study of of small and medium-sized enterprises (SMEs) based on a web-scraping and content analysis of current and archived nanotechnology enterprise web sites. We use this novel approach to explore nanotechnology SMEs transitions from discovery to commercialization and understand how transitions vary by SME characteristics, technology …
A Boosted-Trees Method For Name Disambiguation, Jian Wang, Kaspars Berzins, Diana Hicks, Julia Melkers, Fang Xiao, Diogo Pinheiro
Jan 2012
A Boosted-Trees Method For Name Disambiguation, Jian Wang, Kaspars Berzins, Diana Hicks, Julia Melkers, Fang Xiao, Diogo Pinheiro
Diana Hicks
This paper proposes a method for classifying true papers of a set of focal scientists and false papers of homonymous authors in bibliometric research processes. It directly addresses the issue of identifying papers that are not associated (“false”) with a given author. The proposed method has four steps: name and affiliation filtering, similarity score construction, author screening, and boosted trees classification. In this methodological paper we calculate error rates for our technique. Therefore, we needed to ascertain the correct attribution of each paper. To do this we constructed a small dataset of 4,253 papers allegedly belonging to a random sample …
Performance-Based University Research Funding Systems, Diana Hicks
Dec 2011
Performance-Based University Research Funding Systems, Diana Hicks
Diana Hicks
The university research environment has been undergoing profound change in recent decades and performance-based research funding systems (PRFSs) are one of the many novelties introduced. This paper seeks to find general lessons in the accumulated experience with PRFSs that can serve to enrich our understanding of how research policy and innovation systems are evolving. The paper also links the PRFS experience with the public management literature, particularly new public management, and understanding of public sector performance evaluation systems. PRFSs were found to be complex, dynamic systems, balancing peer review and metrics, accommodating differences between fields, and involving lengthy consultation with …
Powerful Numbers Or A Short Reflection On Influential Analyses In The History Of Science Of Science Policy, Diana Hicks
Dec 2011
Powerful Numbers Or A Short Reflection On Influential Analyses In The History Of Science Of Science Policy, Diana Hicks
Diana Hicks
The quantitative analysis of issues relevant to science policy has a history dating back several decades. Over that time, there have been occasions in which scholarly analyses have escaped from the ivory tower and made an impact on policy discussions or on policy itself. In this paper, I review some of these occasions, looking at what type of analyses were used, who used such analyses, and for what purposes.
Bibliometrics As A Tool For Research Evaluation, Diana Hicks, Julia Melkers
Dec 2011
Bibliometrics As A Tool For Research Evaluation, Diana Hicks, Julia Melkers
Diana Hicks
Creative use of bibliometric analysis in evaluation offers an unparalleled opportunity to take advantage of the rich information embedded in the written products of scientific work to track the output and influence of funded scholars. Many metrics and techniques have been developed: from publication and citation counts to percentile rankings, h-index, impact factor, maps of the knowledge landscape, maps of geographical distribution, and metrics of interdisciplinarity and specialization. Analysis can demonstrate evolution over long periods of time, and can draw quantitative comparisons among subgroups or with others anywhere in the world. It would be dangerous to consider such data and …
Equity And Excellence In Research Funding, Diana M. Hicks, Sylvan Katz
May 2011
Equity And Excellence In Research Funding, Diana M. Hicks, Sylvan Katz
Diana Hicks
The tension between equity and excellence is fundamental in science policy. This tension might appear to be resolved through the use of merit-based evaluation as a criterion for research funding. This is not the case. Merit-based decision making alone is insufficient because of inequality aversion, a fundamental tendency of people to avoid extremely unequal distributions. The distribution of performance in science is extremely unequal, and no decision maker with the power to establish a distribution of public money would dare to match the level of inequality in research performance. We argue that decision-makers who increase concentration of resources because they …
Coverage And Overlap Of The New Social Science And Humanities Journal Lists, Diana Hicks, Jian Wang
Dec 2010
Coverage And Overlap Of The New Social Science And Humanities Journal Lists, Diana Hicks, Jian Wang
Diana Hicks
This is a study of coverage and overlap in second generation social sciences and humanities journal lists with attention paid to curation and the judgment of scholarliness. We identify four factors underpinning coverage shortfalls: journal language, country, publisher size and age. Analysing these factors turns our attention to the process of assessing a journal as scholarly, which is a necessary foundation for every list of scholarly journals. Although scholarliness should be a quality inherent in the journal, coverage falls short because groups assessing scholarliness have different perspectives on the social science and humanities literature. That the four factors shape perspectives …
Structural Change And Industrial Classification, Diana Hicks
Dec 2010
Structural Change And Industrial Classification, Diana Hicks
Diana Hicks
Understanding of structural change is compromised because scholars do not clearly articulate the limits of the classification infrastructure (NAICS or GICS) that shapes empirical analysis. These limits are particularly salient in the study of innovation, an activity that by its nature challenges existing categories. Because innovative industries are often not part of the classification infrastructure, they are invisible in empirical analyses and in government statistics. This paper examines the classification of a population of highly innovative, often small, firms working in gaming devices, packaging, filtration, photonics, imaging, biomedical research and fabless semiconductor design. I find examples of knowledge integration, vertical …
Systemic Data Infrastructure For Innovation Policy, Diana Hicks
Nov 2010
Systemic Data Infrastructure For Innovation Policy, Diana Hicks
Diana Hicks
Progress on the vision laid out in the Science of Science Policy Roadmap requires a move to system level thinking and analysis in the study of technology development. This contribution builds on the insight that this move should be discussed explicitly as it will be a change in practice for the academic community and building a system level infrastructure will encounter foreseeable problems.
As the Roadmap details, agencies have in the past approached outcome measurement individually so a variety of approaches obtain. The interagency task group hopes to move agencies toward a more coordinated approach. A national data infrastructure that …
Towards A Bibliometric Database For The Social Sciences And Humanities, Diana Hicks, Jian Wang
Mar 2009
Towards A Bibliometric Database For The Social Sciences And Humanities, Diana Hicks, Jian Wang
Diana Hicks
In the social sciences, humanities or arts it is largely impossible to substantiate statements on research excellence with reliable indicators for international benchmarking of fields and institutions. To help overcome this limitation, this report examined bibliometric systems in the social science and humanities from the perspective of assessing their potential for institutional research evaluation nationally or internationally.
To assess the feasibility of an adequate bibliometric system in SSH, we must ask: how large is the SSH literature and how much of it should be counted in an evaluation? Working with limited time and resources, our efforts focused on assessing international …
Evolving Regimes Of Multi-University Research Evaluation, Diana M. Hicks
Dec 2008
Evolving Regimes Of Multi-University Research Evaluation, Diana M. Hicks
Diana Hicks
Since 1980, national university departmental ranking exercises have developed in several countries. This paper reviews exercises in the U.S., U.K. and Australia to assess the state-of-the-art and to identify common themes and trends. The findings are that the exercises are becoming more elaborate, even unwieldy, and that there is some retreat from complexity. There seems to be a movement towards bibliometric measures. The exercises also seem to be effective in enhancing university focus on research strategy.
The Effect Of Foreign Science Policy On U.S. Research, Diana M. Hicks
Mar 2008
The Effect Of Foreign Science Policy On U.S. Research, Diana M. Hicks
Diana Hicks
Increased emphasis on strengthening national scientific communities has been seen around the world in recent decades. The US scientific community will feel the effects.
Global Research Competition Affects Measured U.S. Academic Output, Diana M. Hicks
Dec 2006
Global Research Competition Affects Measured U.S. Academic Output, Diana M. Hicks
Diana Hicks
Between 1992 and 1999, the number of papers published by U.S. academics fell by 9 percent as reported in the National Sciences Board’s Science & Engineering Indicators–2002 (SEI). This chapter seeks to understand why this occurred. A 9 percent decline in output could be a valuable tool for advocacy for almost any constituency in U.S. academia. Advocates could report trends in particular fields over limited periods of time to support arguments about the deleterious effects of the emerging patent culture, the insidious effects of health insurers on medical research, the harm of decreasing federal support for engineering, the dangers of …
The Dangers Of Partial Bibliometric Evaluation In The Social Sciences, Diana M. Hicks
Dec 2005
The Dangers Of Partial Bibliometric Evaluation In The Social Sciences, Diana M. Hicks
Diana Hicks
Social science research communities around the world face pressures for quantitative evaluation imposed from outside. Traditional methods of allocating jobs and research funding may not be seen as sufficiently merit-based to ensure research excellence and international competitiveness. In this environment, the preferred evaluation methodology tends to be SSCI-based bibliometrics, more or less exclusively. In this paper, I reflect on the merits of this approach by examining the nature of the social science literature as it relates to bibliometric evaluation. The argument is based on a thorough review of the literature of social science bibliometric methodology.
Bibliometric Evaluation Of Federally Funded Research In The United States, Diana M. Hicks
Jul 2004
Bibliometric Evaluation Of Federally Funded Research In The United States, Diana M. Hicks
Diana Hicks
Research evaluation in the USA historically tended to rely more heavily on peer review than on bibliometric method, but interest in quantitative methods including bibliometrics appears to be growing. In this paper, we discuss the use of bibliometric techniques of research evaluation by the US federal government over the past decade. Within the past decade, commentators have pointed to something of a rebirth of interest in evaluation along with pressure on agencies to develop quantitative indicators. Evaluation of economic and societal outcomes of research has become a priority. Bibliometric method continues to evolve in response to these needs and therefore …
The Four Literatures Of Social Science, Diana M. Hicks
Dec 2003
The Four Literatures Of Social Science, Diana M. Hicks
Diana Hicks
This chapter reviews bibliometric studies of the social sciences and humanities. SSCI bibliometrics will work reasonably well in economics and psychology whose literature shares many characteristics with science, and less well in sociology, characterized by a typical social science literature. The premise of the chapter is that quantitative evaluation of research output faces severe methodological difficulties in fields whose literature differs in nature from scientific literature. Bibliometric evaluations are based on international journal literature indexed in the SSCI, but social scientists also publish books, and write for national journals and for the non-scholarly press. These literatures form distinct, yet partially …
Hospitals: The Hidden Research System, Diana Hicks, Sylvan Katz
Sep 1996
Hospitals: The Hidden Research System, Diana Hicks, Sylvan Katz
Diana Hicks
Using co-authored scientific papers as indicators of research collaboration the pattern of research linkages in the UK during the 1980s is analysed. All sectors collaborate with each other at a rate proportional to publishing size to a first approximation. However, there are deviations from this pattern. Analyzing these deviations, two groups of sectors are found that collaborate with each other more than expected: the GIPU group composed of government, industry, polytechnics and universities, and the HSNR group composed of hospitals, special health authorities (SHAs), non-profit organisations and research councils. This suggests that a biomedical innovation system co-exists with the more …
Published Papers, Tacit Competencies And Corporate Management Of The Public/Private Character Of Knowledge, Diana Hicks
Dec 1994
Published Papers, Tacit Competencies And Corporate Management Of The Public/Private Character Of Knowledge, Diana Hicks
Diana Hicks
This paper focuses on the movement of scientific and technological knowledge. It explores companies' reasons for publishing in the scientific and technical literature, reasons that turn on the need to link with other research organisations. The analysis begins by establishing that firms do indeed publish. Such publishing mediates links with other organisations, serving to signal the presence of tacit knowledge and to build the technical reputation necessary to engage in the barter-governed exchange of scientific and technical knowledge. Similar processes are seen in other areas of technical knowledge exchange.
