Education Commons^™

Welcome Address, Baron Wolf

Research Analytics Summit 2024

No abstract provided.

Leveraging Herd Data To Understand Institutions’ Competitive Edge, Growth Potential, And Strategic Collaborations, Hanin Alhaddad, Joshua Roney

Research Analytics Summit 2024

This presentation delves into understanding institutional strengths, weaknesses, and areas of growth potential within the landscape of higher education. Leveraging data from the Higher Education Research (HERD) survey, we analyze research expenditures in ten broad scientific fields and forty sub-detailed research fields across higher educational institutes in the United States. We utilize macro and microeconomics methodologies to analyze and identify research specializations by institution, along with research fields exhibiting high growth potential. This comprehensive approach aims to offer valuable insights for strategic planning, fostering smart collaborations, and guiding investments to nurture growth within an institution.

Exploring Research Metadata Governance With Design Thinking, Scott Woods, Natalie Simonson

Research Analytics Summit 2024

If you want great research analytics, you’re going to need great data governance for your research metadata. That’s a tall order when the best information often spans incompatible systems, departments, policies, and mindsets.

Research metadata governance is a great illustration of how research analytics is not just a technology problem. It happens at the intersection of technology, policy, process, and culture. Do any of us feel like our organization is “nailing” metadata governance? What level of research analytics could you achieve if there could be more alignment and cooperation around this kind of governance?

In this session, you will join …

Bringing Artificial Intelligence To Research Analytics Research Highlighter-Matchmaker At Suny Ualbany, Tianning Huang, Yuemin Li

Research Analytics Summit 2024

Research analytics has become an indispensable part in today’s higher education institutions’ organizational structure. It not only provides a mechanism or platform to present research awards and expenditures data to internal and external data users, but also serves a superior tool that enables data-driven strategic decision makings with more reliable, valid, and in-depth analysis. The development of Artificial Intelligence techniques could push the boundary of Research Analytics to an even higher level by incorporating the abundance of data scattered across the university. We propose to present one attempt adopted by University at Albany, State University of New York (UAlbany) to …

Where Do I Start?, Andrea Crystelle, Lori Schultz

Research Analytics Summit 2024

In this session, we will start at the beginning and talk about what you need to think about in setting up research analytics capabilities for your organization. There are so many options for tools and other resources to get started, but what if you only have Excel? How do you get reporting off the ground when it hasn’t been done (or done consistently) at your workplace? We’ll talk about ways to get started, including who you need to talk with, what you need to consider, and how to move forward with basic reporting.

You Are What You Eat ... And Enter Into Data Fields, Hansa Magee, Nick Bilyk

Research Analytics Summit 2024

No abstract provided.

Representing Your Institution Through Data, Lou Hurst

Research Analytics Summit 2024

In this session, we will consider how to use place-based data to build your case to sponsors for funding research at your institution, particularly for sponsors who operate on the national or international scale. The setting of your institution—the communities it developed in, the region where it operates, and the people it reaches and serves—is key for conveying its unique capacities and potentials, and for making sponsors eager to bring you into their funding portfolio. How can data help you introduce yourself as an institution and tell your story in geographical and economic context? In this session we will explore …

Generating Reliable Collaboration Data: A Proof-Of-Concept Project Using Research Administration Data, Christopher Perkins, Julie Zappi

Research Analytics Summit 2024

Central administration at Rutgers University requested help from the Office for Research to develop a proof-of-concept tool to count collaborations between faculty, departments, schools and chancellor units across the university. This presentation will discuss how the Data, Analytics and Business Intelligence team focused on research administration and research output data to define a “collaboration,” structure available data, create interactive visualizations and allow end users to customize the level of detail displayed.

This presentation will begin by discussing the importance of generating reliable collaboration data in a university setting including use cases for the data. It will then describe the dataset …

Collabnext -A Person-Focused Open Knowledge Graph For Collaborations With Emerging Researchers, Lew Lefton

Research Analytics Summit 2024

CollabNext is being developed as part of the NSF ProtoOKN (Prototype Open Knowledge Network) effort. This project originated as a partnership between Georgia Tech and the Atlanta University Center, and is now being developed jointly by Fisk University, Georgia Tech, Morehouse College, Texas Southern University, and University at Buffalo with support from the NSF TIP Directorate.

Our goal is to develop a knowledge graph based on people, organizations, and research topics. We are adopting an intentional design approach which initially prioritizes HBCUs and emerging researchers in a deliberate effort to counterbalance the Matthew effect, a naturally accumulated advantage of well-resourced …

Ramp Research Analytics Mentorship Program, Andrea Christelle, Lou Hurst, Baron Wolf

Research Analytics Summit 2024

No abstract provided.

The Balancing Act: Data Analytics And Radical Invention, Andrea Christelle, Baron Wolf

Research Analytics Summit 2024

Metrics and analytics are becoming indispensable tools for guiding decision-making and resource allocation. That’s why we are all at this conference. However, what happens when the path to innovation veers off the beaten track, to such an extent that metrics don’t capture it? This session will delve into the delicate balance between relying on metrics and fostering revolutionary invention by exploring how research administrators can strategically harness analytics while also nurturing the creative space necessary for groundbreaking discovery.

This thought-provoking session will engage participants in a dialog that considers the unanticipated nature of invention, and how it can be fostered, …

Evaluating A Pi Policy With Data, Lori Ann M. Schultz

Research Analytics Summit 2024

At our organizations, policies are often implemented then never reviewed again. We often don’t know why the policy existed in the first place. In this session, we will discuss how the data we already collect can be used to evaluate existing policies, proposed policies, and potential changes. We will walk through an example of how a proposed institutional policy was examined through the use of data on research proposals and awards, and answer the question “Do we really need this policy?” Bring your examples to talk through options for policy evaluation!

A Data Driven Approach To Research Enterprise Growth And Infrastructure Design, Marisa Zuskar, Gabby Labayen

Research Analytics Summit 2024

As proposal, subaward, award, and agreement volumes continue to grow at your institution, how can you strategize to support the size of your research infrastructure? How can you justify the expansion of your teams and talent? How do you adjust roles and workload to align with the growing portfolio? What technology considerations should you make to track this?

This session will cover how to leverage data to make effective business decisions regarding resource needs and allocation methodology to meet growing demands, starting with your internal data and then looking externally. We will cover strategies for using data analytics to efficiently …

Using Agency Data For Research Development, Lou Hurst

Research Analytics Summit 2024

This session will explore ways to locate, analyze, and present sponsor-provided data to guide research development. Individual investigators, who may be proposing to a funder for the first time, often want to know about funding success rates, as well as more nuanced insight into the kinds of projects agencies tend to favor. They may also benefit from insight into changing workforce demographics and career trajectories in their discipline. At the departmental and institutional level, leadership seeks information about trends in funding availability and mechanisms, and in patterns of grants made, to better position themselves for these opportunities. As a research …

Navigating Faculty Activity Reporting: Maximizing Transparency And Efficiency, Kathy Felts, Michael Katz, Jenny Hulsen

Research Analytics Summit 2024

Join us as we embark on a journey through the intricacies of faculty activity reporting, exploring strategies to maximize transparency and efficiency. This presentation will dive into the essential components of faculty activity reporting, examining the benefits/challenges, and providing best practices. Discover how leveraging Academic Analytics can streamline workflows, enhance data accuracy, and save faculty time and effort.

Co-Authorship Maps To Support Leadership Selection, Robert Hc Chen

Research Analytics Summit 2024

VOSViewer co-authorship mapping is a powerful tool typically used for analyzing research collaboration. Users provide publication data and VOSViewer produces a map where authors are plotted on a 2-dimensional map based on how often they are in the author lists of the same publication.

In this presentation, I propose a series of tweaks to the input data that can leverage co-authorship maps to support leadership selection based on how often candidates co-author papers with their institutional peers and some of the attributes of these papers. I will suggest how best to interpret the resulting maps and address the major assumptions …

Answering Big Questions With Big Data: Using Analytics Across The Research Ecosystem, Ann Beynon

Research Analytics Summit 2024

The research landscape continues to shift and evolve. With challenges around research integrity and security growing, research leaders must rely on the highest quality data to make confident decisions. Universities and funders need to understand the societal impact of research investments in addition to the scholarly impact. Has research helped to advance the UN Sustainable Development Goals? What are the emerging research areas that will lead to commercial innovations and sustainability advancements? We will discuss how quantitative data from Web of Science and InCites can help research stakeholders understand these big questions and inform their strategic research decisions.

Research Analytics Maturity Model & Herd: Transforming Tomorrow By Using Data To Drive Decisions, Baron Wolf, Katherine Leung Robershaw

Research Analytics Summit 2024

No abstract provided.

Finding And Hiring A Research Analyst Unicorn, Hansa Magee, Nick Bilyk

Research Analytics Summit 2024

As more data analytics programs pop up, we see graduates with these degrees and certificates in our applicant pool. However, the analyst that works in higher ed administration is not like a data scientist that looks for biomarkers in genomic data or the programmer that writes in R all day. What are the skills needed for this job? What exactly is this job that we are hiring for? Should managers prioritize applicants with analytics certificates? Or is experience with sponsored projects more important? We (your presenters) don’t have perfect answers but we (the collective conference attendees) can come up with …

The Predictive Role Of Grapho-Morphological Knowledge In Reading Comprehension For Beginning-Level L2 Chinese Learners, Tianxu Chen, Sihui Ke, Keiko Koda

Modern and Classical Languages, Literatures and Cultures Faculty Publications

Reading comprehension entails a set of distinct, yet interdependent cognitive, linguistic, and nonlinguistic processes. Previous second language (L2) Chinese studies have identified significant and positive impacts of grapho-morphological knowledge at the character and subcharacter (radical) levels on passage reading comprehension; however, little is known regarding how early L2 grapho-morphological knowledge at the character and radical levels jointly predict later L2 reading comprehension. This study aimed to fill this gap. One hundred and five beginning-level L2 Chinese collegiate learners were recruited, and completed two character-related and two radical-related tasks in Week 8, as well as one reading comprehension tasks in Week …

Differential Item Functioning Among English Language Learners On A Large-Scale Mathematics Assessment, Ruixue Liu, Kelly D. Bradley

Educational Policy Studies and Evaluation Faculty Publications

The current study research showed the nature and potential sources of the gaps in mathematics achievement between English language learners (ELLs) and non-English language learners (non-ELLs). The nature of achievement gap was examined using three DIF methodologies: including Mantel-Haenszel procedure, Rasch model, and Hierarchical Generalized Linear Model (HGLM). These were conducted at the item level in contrast to total test level. Results revealed that the three DIF approaches identified 10 common items. These 10 items demonstrated in favor of non-ELLs. Findings from this study will help educational researchers, administrators, and policymakers understand the nature of the achievement gap in mathematics …

Pace Yourself: Impact Of Covid-19 On Patient-Centered Care Experience, Kristen Wilhite, Mikael D. Jones, Clark D. Kebodeaux

Pharmacy Practice and Science Faculty Publications

(1) Background: The outbreak of the novel coronavirus, COVID-19, forced colleges of pharmacy to implement new online learning methodologies to ensure that students could complete required courses. This transition was especially acute for laboratory simulation courses that require students to practice professional skills. This study aims to compare student assessment performance within a simulation-based laboratory course for students who completed the module prior to and after the online transition. (2) Methods: This study was a retrospective cohort comparison of student outcome performance with two distinct content delivery methods. Students were organized into two tracks at the beginning of the semester …

Making A Tough Choice: Teacher Target-Setting And Student Achievement In A Teacher Performance System Using Student Learning Objectives, Allison Atteberry, Sarah E. Lacour

Educational Policy Studies and Evaluation Faculty Publications

The use of student learning objectives (SLOs) as part of teacher performance systems has gained traction quickly in the United States, yet little is known about how teachers select specific students’ learning goals. When teachers are evaluated—and sometimes compensated—based on whether their students meet the very objectives the teachers set at the start of the year, there may be an incentive to set low targets. SLO systems rely on teachers’ willingness and ability to set appropriately ambitious SLOs. We describe teachers’ SLO target-setting behavior in one school-district. We document the accuracy/ambitiousness of targets and find that teachers regularly set targets …

The Past And Future Of Teacher Efficacy, Thomas R. Guskey

Educational, School, and Counseling Psychology Faculty Publications

Knowing how the concept of teacher efficacy has evolved in education research shows us good ways to raise teachers' confidence now.

Best Practices For Research Analytics & Business Intelligence Within The Research Domain, Baron Wolf, Terri Hall, Katherine Robershaw

Research Analytics Academic Publications

While an increased number of industries with business operations have been shaping their use of data analytics, the use of such tools and methods within the higher education research domain, specifically research administration, is still in its infancy. This mixed-methods study collected data to identify best practices in how universities and other research organizations use data analytics to drive their strategic agendas, create efficiency in operations, and promote complex research proposals throughout their institutions. Research methods included a survey to collect data on how research offices are using analytics and business intelligence tools, Rasch analysis (Rasch, 1993) to examine survey …

The Impact Of Standards-Based Learning: Tracking High School Students’ Transition To The University, Thomas R. Guskey, Matt Townsley, Thomas M. Buckmiller

Educational, School, and Counseling Psychology Faculty Publications

This study sought to determine if the implementation of standards-based learning in high schools affects students’ transition to learning in university courses. Surveys and interviews with 13 students who had graduated from high schools implementing standards-based learning and who had completed their first academic semester at a midsize, private, Midwest university revealed no detrimental effects. The most frequently mentioned transition difficulties related to social issues and time management. Implications for implementing high school grading reforms are discussed.

Chart A Clear Course: Evaluation Is Key To Building Better, More Relevant Learning, Chase Nordengren, Thomas R. Guskey

Educational, School, and Counseling Psychology Faculty Publications

When we engage in professional learning, we do it for one big reason: to get better at supporting students. Rigorous and thoughtful program evaluations can provide the critical connection between well-designed programs or initiatives and continuous improvement that builds essential knowledge and skills for educators. Evaluation helps us examine what has been accomplished in a professional learning initiative and identify course corrections that can help the initiative improve.

A Digital Ethnography Of Teach For America: Analysis Of Counternarrative From The Truth For America Podcast, Julian Vasquez Heilig, T. Jameson Brewer, Amber K. Kim, Miguel Sanchez

Educational Policy Studies and Evaluation Faculty Publications

To analyze the counternarrative in the public discourse surrounding Teach For America (TFA), this paper represents the first digital ethnography in education policy. We conduct a qualitative analysis of Truth For America, an education policy podcast. We found four overarching themes that arose from conversations with respondents: (1) problematic practice, preparation, and pedagogy; (2) concerns linked to critiquing TFA and the organization’s responses to that critique; (3) issues related to race and diversity; and (4) disconcerting funding practices and political power. We conclude by discussing the implications of how individual-level stakeholder experiences inform the public discourse about TFA.

Applying The Rasch Model To Evaluate The Self-Directed Online Learning Scale (Sdols) For Graduate Students, Hongwei Yang, Jian Su, Kelly D. Bradley

Educational Policy Studies and Evaluation Faculty Publications

With the rapid growth of online learning and the increased attention paid to student attrition in online programs, much research has been aimed at studying the effectiveness of online education to improve students’ online learning experience and student retention. Utilizing the online learning literature as a multi-faceted theoretical framework, the study developed and employed a new survey instrument. The Self-Directed Online Learning Scale (SDOLS) was used to examine graduate student perceptions of effectiveness of online learning environments as demonstrated by their ability to take charge of their own learning, and to identify key factors in instructional design for effective improvements. …

Eight Essential Principles For Improving Grading, Susan Brookhart, Thomas R. Guskey, Jay Mctighe, Dylan Wiliam

Educational, School, and Counseling Psychology Faculty Publications

Done well, grading can play a key role in a balanced district assessment system.