Education Commons^™

The Balancing Act: Data Analytics And Radical Invention, Andrea Christelle, Baron Wolf

Research Analytics Summit 2024

Metrics and analytics are becoming indispensable tools for guiding decision-making and resource allocation. That’s why we are all at this conference. However, what happens when the path to innovation veers off the beaten track, to such an extent that metrics don’t capture it? This session will delve into the delicate balance between relying on metrics and fostering revolutionary invention by exploring how research administrators can strategically harness analytics while also nurturing the creative space necessary for groundbreaking discovery.

This thought-provoking session will engage participants in a dialog that considers the unanticipated nature of invention, and how it can be fostered, …

Exploring Research Metadata Governance With Design Thinking, Scott Woods, Natalie Simonson

Research Analytics Summit 2024

If you want great research analytics, you’re going to need great data governance for your research metadata. That’s a tall order when the best information often spans incompatible systems, departments, policies, and mindsets.

Research metadata governance is a great illustration of how research analytics is not just a technology problem. It happens at the intersection of technology, policy, process, and culture. Do any of us feel like our organization is “nailing” metadata governance? What level of research analytics could you achieve if there could be more alignment and cooperation around this kind of governance?

In this session, you will join …

Bringing Artificial Intelligence To Research Analytics Research Highlighter-Matchmaker At Suny Ualbany, Tianning Huang, Yuemin Li

Research Analytics Summit 2024

Research analytics has become an indispensable part in today’s higher education institutions’ organizational structure. It not only provides a mechanism or platform to present research awards and expenditures data to internal and external data users, but also serves a superior tool that enables data-driven strategic decision makings with more reliable, valid, and in-depth analysis. The development of Artificial Intelligence techniques could push the boundary of Research Analytics to an even higher level by incorporating the abundance of data scattered across the university. We propose to present one attempt adopted by University at Albany, State University of New York (UAlbany) to …

Generating Reliable Collaboration Data: A Proof-Of-Concept Project Using Research Administration Data, Christopher Perkins, Julie Zappi

Research Analytics Summit 2024

Central administration at Rutgers University requested help from the Office for Research to develop a proof-of-concept tool to count collaborations between faculty, departments, schools and chancellor units across the university. This presentation will discuss how the Data, Analytics and Business Intelligence team focused on research administration and research output data to define a “collaboration,” structure available data, create interactive visualizations and allow end users to customize the level of detail displayed.

This presentation will begin by discussing the importance of generating reliable collaboration data in a university setting including use cases for the data. It will then describe the dataset …

From Policy Sources To Research Information Management Sources: Incorporating Multiple Bibliographic Data Sources Into Dashboard Analytics, Rachel Miles

Research Analytics Summit 2024

As the availability of bibliographic data becomes more diverse and unique, so do the complexities of those sources. As a result, it can be challenging and daunting to analyze such data, especially from a single research analytics service request. In this session, the presenter will demonstrate examples of building dashboards to analyze data for research outputs of a college using data from a research information management system, a policy database, and a bibliographic database. Attendees will learn about the advantages of using RIM systems to perform analytics as well as their shortcomings; they’ll also learn how to export data from …

Using Public Policy Citation Data In University Decision Making: Understanding, Sharing And Responsible Use, Euan Adie

Research Analytics Summit 2024

The 2018 Evidence Act has mandated US government departments and agencies to be much clearer about their research agendas and the engagement they want with academic researchers and institutions. Tracking those engagements has become easier as data providers increasingly supply information on policy engagement or policy citations, tracking where academics at an institution have been cited in or mentioned by name in government reports and policy documents at a local, state or federal level. This session covers the processes, policies and best practice you need to put in place to start using policy engagement data responsibly, both to track and …

Radtrials: Clinical Trial Startup Tracker At Radiology, Anna Romanowska-Pawliczek

Research Analytics Summit 2024

Clinical study startup is one of the most time and resource-intensive parts of the clinical trials. Its impacts are long-lasting, as delays can affect the study conduct. For study startups, 85% of the industry still relies on Excel spreadsheets. But spreadsheets are passive – they don’t allow you to proactively manage activities and don’t tell you where to focus your attention. We designed, implemented, and now fully adopted a new comprehensive workflow centered around a Smartsheet-based solution with automated reports and dashboards for clinical research coordinators (CRCs), managers, faculty, and leadership. We seek to standardize the startup process across the …

Open Alex, Kyle Demes

Research Analytics Summit 2024

Research organizations of all shapes and sizes need information on the research activities at their institutions and other institutions around the world to effectively plan, implement, and evaluate their research strategies. Historically, that information was only accessible through closed commercial licenses. This session will introduce OpenAlex, the most comprehensive and only completely open index of the world’s research ecosystem. The presentation will first provide an overview of OpenAlex, highlighting the importance of openness and transparency in research analytics, and will walk-through how to perform common research analyses in OpenAlex.

Unlocking Opportunities: Leveraging Generative Ai In University-Sponsored Programs Operations, Lisa A. Wilson, Tubal Yisrael, Alex Wagner

Research Analytics Summit 2024

Explore the transformative potential of generative AI in university-research administration. As universities strive to enhance research capabilities and support a culture of innovation, the need for efficient and effective management of sponsored programs has become paramount. This presentation will share lessons from deploying a sponsored programs’ Large Language Model Chatbot and how it optimizes research administration operations and unlocks opportunities. By harnessing the power of GenAI, a university office of sponsored programs chatbot can develop training materials, policies and SOPs. It can offer immediate support and guidance by analyzing queries and providing real-time responses, empowering staff members to overcome challenges …

Generating Impact Narratives With Ai, Sean Newell, Charity Elifritz

Research Analytics Summit 2024

Senior academic leaders face mounting pressure to communicate academic impact to various stakeholders, including faculty, students, funding bodies, government agencies, and the media. However, assessing impact extends beyond traditional bibliometrics, encompassing influence on policy, culture, environment, and technology, making it complex to define and measure. Moreover, the interpretation of research impact can vary significantly depending on the perspective. In this discussion, we explore how Elsevier leverages AI to integrate quantitative metrics and qualitative narratives, offering a comprehensive approach to addressing the nuanced challenge of understanding and quantifying research impact. Through real-world examples, we demonstrate the efficacy of this approach and …

A Mixed Methods Study Of Efficacy And Digital Learning Competencies For School Administrators In A Rural North Carolina District, Stephanie B. Harrell

Doctor of Education Dissertations

The purpose of this mixed methods research study was to determine the efficacy of digital learning competencies for school administrators in a rural eastern North Carolina district. The study had four research questions that were answered through a survey and a focus group. The survey had 44 quantitative items that participants chose from a 4-point Likert scale and two open-ended questions. A total of 20 administrators from District A completed the survey. A focus group with four participants was conducted following the closing of the survey to gain a better understanding of the results from the survey. Previous research in …

Interviewing K-12 Education Experts And Elites, Jose Eos Trinidad

Interdisciplinary Studies Faculty Publications

Understanding education experts and elites is crucial in the context of their larger influence on education and the public’s greater skepticism and criticism of their work. This paper distinguishes between traditional and expert/elite interviews (EEIs), and highlights strategies for conducting them. Experts and elites have relatively broader influence, more synthesized but less situated knowledge, more embedded professional networks, and less anonymity than the lay public—and interviews need to adjust to these differences. To do so, researchers should consider strategies for (1) access, (2) trust, (3) preparation for interviews, and (4) asking sensitive and awkward questions in contexts of significant power …

Exploring Black Queer Doctoral Student Experiences With Utilizing Campus Services, Mitchell Everett

Electronic Theses and Dissertations

This study utilized narrative inquiry to examine the experiences of Black Queer Doctoral Students (BQDS) with campus services and their ability to ameliorate minority stress and establish community with other students minoritized by their sexual or gender identity. I used the minority stress model and intersectionality as frameworks to understand how students minoritized by their race and sexual identities experienced campus services. The minority stress model provided an explanation of the stress BQDS may experience due their minority identity (Meyer, 2003, 2013). Connecting to community is also an ameliorating factor in reducing minority stress. In addition, structural intersectionality addressed the …

The Impact Of The Science Of Reading Required Training On The Reading Proficiency Levels Of Third-Grade African American Male Students, Tonya Campbell

Doctor of Education Dissertations

Proficiency in reading is an essential skill for achieving success in the modern era, and African American students have struggled with reading due to insufficient support during literacy instruction. Through the implementation of the Science of Reading™ course, this research aimed to examine the effect of the Language Essentials for Teachers of Reading and Spelling (LETRS™) curriculum on the success of African American third-grade males based on the South Carolina College- and Career-Ready (SC READY™) summative assessments and Northwest Academic Evaluation Measures of Academic Progress (NWEA MAP™) data. This study followed the year before LETRS™ was introduced (2019), the 2 …

The Impact Of Teacher Preparedness And Professional Development On Fourth-Grade Students' Science Achievement, Craig L. Mayo, Faye Bradley

Journal of Research Initiatives

Science scores among US fourth-grade students have declined compared to their international counterparts in recent years. Recent results show that teachers are the most impactful influence on student success and accountability. Teacher preparedness and professional development are two key areas that serve as indicators of providing relevant and essential information for students' success. A correlational quantitative study was conducted to assess the relationship between teacher preparedness and professional development on fourth-grade students’ science achievement. The TIMSS 2019 data were secured from the Boston College, TIMSS, and PIRLS International websites. The data was evaluated using the SPSS 27 Hierarchical Linear Regression. …

An Analysis Of Interaction Patterns In Students' Research Essay Guidance To Build Six Main Parameters Of The Indonesian National Qualification Framework: Covid-19 Pandemic Period, Undang Rosidin, M. Setyarini, Widyastuti Widyastuti

Jurnal Penelitian dan Evaluasi Pendidikan

This study aims to reveal the interaction patterns and inhibiting factors for the final project guidance in the Mathematics and Natural Sciences Education Department, Universitas Lampung, to develop the six main parameters of the Indonesian National Qualifications Framework (INQF). The study employed a mixed method involving 67 students who composed the final project for the 2021/2022 academic year selected from 314 final project students using the cluster random sampling technique. The data in this study were primary and secondary data obtained by questionnaires, interviews, and documents, analyzed qualitatively, quantitatively, and descriptively. The closed-response questionnaire was used regarding the interaction pattern, …

Measuring Students' Generic Skills Through National Assessment, Toha Makhshun, Bassam Abul A'La, Kusaeri Kusaeri

Jurnal Penelitian dan Evaluasi Pendidikan

This study aims to measure the generic skills (GS) of Madrasah Aliyah (MA) students in Indonesia. Respondents who participated in this study were madrasah students scattered in East Java and Central Java as representatives of MAs in Indonesia. The method used in this research is a survey distributed via Google Form. Three dimensions are mentioned in the research instrument for measuring GS: religious moderation, critical and creative thinking skills, and interpersonal skills. Using SPPS version 16 software, a descriptive analysis was employed as the data analysis technique. The results of this study showed that the GS of students with religious …

Development Of A Scientific Attitude Measurement Instrument For Dharmacariya Department Students At Stabn Using The Fishbein And Ajzen Model, Rahmat Syah, Daddy Darmawan, Syaiful Syamsuddin

Jurnal Penelitian dan Evaluasi Pendidikan

This study applies the Fishbein and Ajzen model to develop a Scientific Attitude Measurement Instrument for Students, focusing on attitudes, beliefs, and intentions towards scientific behaviors. The quantitative research employs an instrument development method, initially constructing dimensions reflecting various scientific attitudes such as attitudes towards the scientific method, collaboration, openness to knowledge, criticality, respect, honesty, independence, and interest in data analysis. The instrument measures students' scientific attitudes across three main dimensions: attitude, subjective norms, and behavioral control, aligning with the Fishbein and Ajzen model. It covers key aspects like attitudes towards the scientific method, emphasizing rigorous approaches like Popper's Falsification …

Developing And Implementing An Instrument For Assessing Critical Thinking And Visual Representations In Learning Physics Materials Of Optical Instruments, Dewi Nurulhasni, Eka Ayu Nurbaiti, Rosa Safirotun Nabila, Jumadi Jumadi

Jurnal Penelitian dan Evaluasi Pendidikan

This study aims to develop an assessment instrument to measure students' critical thinking and visual representation abilities in optical instrument material. This research and development (R&D) employed the ADDIE (Analysis, Design, Development, Implementation, and Evaluation) instructional model. This research develops a test instrument that has more specific stages so the ADDIE development stage is integrated with the development stage of the test instrument developed by Mardapi. The subjects in this study are grade XI students of State Madrasah Aliyah (MAN) 4 Bantul. This study produced a test instrument for measuring students' critical thinking and visual representation abilities, with high validity,and …

Analysis Of Students' Perceptions On The Freedom Of Learning - Independent Campus (Mbkm) Program At Chemistry Study Program Of Ma Chung University, Yuyun Yuniati, Lilis Lestari Wilujeng

Jurnal Penelitian dan Evaluasi Pendidikan

Considering that Indonesian education has gone through such a long journey from 1922 to 2022 now, it is necessary to have an educational program that is able to meet the demands of workers' needs in this era of globalization. In the end, the Directorate General of Higher Education, Research, and Technology designed an education policy system in the realm of higher education, namely "Freedom of Learning - Independent Campus", as an accurate strategy for optimizing academics in today's Indonesia. In this study, a survey was aimed at 40 undergraduate students of the Chemistry Study Program at Ma Chung University. Data …

Item Quality Analysis Using The Rasch Model To Measure Critical Thinking Ability In The Material Of The Human Digestive System Of Biology Subject In High School, Wanda Agus Prasetya, Anggi Tias Pratama

Jurnal Penelitian dan Evaluasi Pendidikan

This study aims to determine the quality of the Biology instrument items on the digestive system material with the Rasch model for analyzing critical thinking skills. The research employed a quantitative descriptive method involving 63 students of senior high schools in Yogyakarta. The data were collected using a critical thinking skills description test and processed using the Rasch model with the Winstep 5.0.3.4 program. The study shows that the overall validity is acceptable. The item validity did not require improvement in items 14, 3, 8, 13, 12, 1, 4, 10, 6, 2, 7, 11, 15, and 9, and required improvement …

Dominant Factors That Determine College Students Completing Studies In Mathematics Education Study Programs, Sintha Sih Dewanti, Aji Joko Budi Pramono

REID (Research and Evaluation in Education)

Failure to graduate on time is a problem for both the university and the students themselves. Beside the competencies possessed by students, many other factors affect the completion of student studies. This study aims to reduce the variables that really affect the completion of student studies. The approach of this research was survey research on students of the Mathematics Education Study Program at least in semester 7 (currently taking a final project course). There are 17 factors that determine the completion of student studies, namely achievement motivation, discipline, interest, intelligence, study habits, health, part-time work activities, organizational activities, curriculum, mentoring …

The Implementation Of Geography Learning With Spatial Representation Using The Discrepancy Evaluation Model, Muhammad Nursa'ban, Mukminan Mukminan

REID (Research and Evaluation in Education)

This study aims to evaluate the learning process of geography with spatial representation in high school using the Discrepancy Evaluation Model (DEM) from Provus. Evaluation goes through five stages: design, installation, process, and product, as well as implementation comparison. Respondents were five teachers and 208 students from SMA 1 Srandakan, MAN 2 Yogyakarta, and SMA 1 Pundong in Yogyakarta. The evaluation was carried out in February-July 2021. The focus of the evaluation includes (1) learning design, (2) learning tools, (3) learning implementation, (4) learning outcomes, geographic critical thinking, and (5) implementing learning comparisons. Data were collected through document studies, observations, …

Evaluation Of Music Learning During The Covid-19 Pandemic In Senior High Schools, Putra Andino Nugrahhu

REID (Research and Evaluation in Education)

The purpose of this study is to see the extent to which the implementation of teaching and learning activities for Cultural Arts subjects, especially in the field of Music Arts at high schools in Palangka Raya, was carried out during the COVID-19 pandemic. This research is evaluation research with a descriptive qualitative approach that uses a discre­pancy evaluation model that measures the gaps in the learning process. Data was collec­ted through observation, interview, and documentation techniques and validated using triangulation and source triangulation. The results of the study found that in the evalua­tion of input, music art learning consisting of …

Development And Validation Of A Self-Assessment-Based Instrument To Measure Elementary School Students' Attitudes In Online Learning, Ari Setiawan, Wiputra Cendana, Mark Ayres, Azim Abdurakhmanovich Yuldashev, Sri Panca Setyawati

REID (Research and Evaluation in Education)

Online learning during the COVID-19 pandemic makes it difficult for teachers to assess student learning attitudes. Limited availability of instruments to measure attitudes of students when they are engaged in online learning leads to difficulty of teachers to conduct appropriate assessments on that measure. The current study, therefore, mainly was intended to produce a self-assessment-based instrument that is feasible to use to measure students' attitudes in online learning. In order to produce such instrument, we used developmental research method by following steps in the instrument design that is proposed by McCoach, Gable, and Madura. Furthermore, in order to provide feasibility …

Item Analysis Of Reading Comprehension Questions For English Proficiency Test Using Rasch Model, Henda Harmantia Dewi, Siti Maftuhah Damio, Sukarno Sukarno

REID (Research and Evaluation in Education)

The need to take English as a foreign language proficiency test (known as TOEFL [Test of English Language Proficiency]) has been gaining popularity in Indonesia. The increasing demands for such a test and its expensive cost have reinforced many institutions to develop TOEFL instruments and administer the test internally. However, constructing a test instrument is a complex process that makes conducting item analysis become more challenging. Meanwhile, item analysis is crucial to assess the items' quality. Therefore, this study reported the results of statistically analyzing 20 questions of TOEFL reading comprehension that were analyzed in terms of the test reliability, …

Developing A Religiosity Scale For Indonesian Muslim Youth, Shodiq Abdullah, Warsiyah Warsiyah, Ju'subaidi Ju'subaidi

REID (Research and Evaluation in Education)

This study aims to construct and test the validity of the Islamic youth religiosity scale. The population in this study is Muslim students of senior high schools in Surakarta, Central Java, with a sample of 258 established using the random sampling technique. The data analysis used the Linear Structural Model. The result shows that the RMSEA (‰¤ 0.08) and GFI (‰¥ 0.09) values from the four dimensions (belief, ritual, social, commitment meet the standard values of compatibility with the respective values for RMSEA beliefs = 0.055 GFI = 0.94, RMSEA rituals = 0.026 GFI = 0.99, social RMSEA = 0.059 …

Determinant Factors Affecting The Research Performance Of Lecturers Receiving External Funds, Haryanto Haryanto, Nur Kholis, Samsul Hadi, Ezi Apino, Muh. Asriadi Am, Cerry Kartika Trizkyana

REID (Research and Evaluation in Education)

Quality research can positively impact the development of science and technology and improve the reputation of universities. This study investigates the determinant factors that affect the research performance of lecturers who obtain external funding. The sample was 41 lecturers who obtained external research funds from various institutions (e.g., Directorate of Research, Technology and Community Service, Ministry of Education, Culture, Research and Technology of the Republic of Indonesia; National Research and Innovation Agency of the Republic of Indonesia; and similar institutions) for the last three years (2021 - 2023). Cluster analysis using secondary data (Science and Technology Index or SINTA data) …

Boundaries Of Empirical Approaches In Educational Research, Christopher Olusola Omoregie

Journal of Research Initiatives

This paper critically reviews the research done in education faculties in Nigerian universities. This research, though categorized in postgraduate schools or colleges as mainly in the liberal arts/humanities and the social sciences, depends on the theories and methodologies from other disciplines. The arts and social sciences are disciplines where undergraduates in education take courses in teaching to earn bachelor’s degrees, the postgraduate level offers varied opportunities for educational research to maximize the uniqueness of mixed method research for education.

Instrument And Evaluation Of Students' Involvement In Learning Mathematics In 21-Century Skills At Senior High Schools Of Riau Province, Dedek Andrian, Leo Adhar Effendi

Jurnal Penelitian dan Evaluasi Pendidikan

Student engagement is the most crucial factor in learning activities because it is the main factor in improving student learning outcomes in class. This study aims to develop instruments and evaluate student involvement in learning mathematics to achieve maximum learning outcomes at school. The population in this study was all high school students in Riau Province. The sample is 150 high school students selected by multiple-stage random sampling based on high, medium, and low student achievement levels. Data analysis in this study was carried out four times: (1) data analysis based on the results of expert assessment of the instrument …