Educational Assessment, Evaluation, and Research Commons^™

Digital Stories Of Resilience: A Case Study Of Urban Community College English Learners, Patricia George Hunter

Journal of English Learner Education

This study utilizes a qualitative case study approach to learn how project-based, digital storytelling (DST) voices the resilience of post-secondary English language learners (ELLs) attending a minority-serving, open-access, community college in New York City. Along with many studies that have examined the usefulness of DST in oral production, this research investigates whether project-based digital storytelling enhances students’ understanding of academic content related to resilience and conventional composition. Narrative inquiry (NI) research technique is the basis of the digital story project in an English as a Second Language (ESL) course. In this case, the study relies on observations and artifacts to …

Game-Based Learning With Augmented Reality, Kantida Nanon

Dissertations

Augmented Reality (AR) technology provides an effective and controllable learning environment, especially for complicated, potentially dangerous, or critical content. Effective implementation of interaction and platforms for educational tools can maximize learner engagement and optimize the learning experience. This dissertation explores the potential of AR technology to enhance learning experiences across multiple platforms and subject areas, particularly within chemistry and mathematics. The dissertation is grounded in the field of Human-Computer Interaction (HCI) and explores the potential of AR to increase learner engagement, improve learning outcomes, and transform traditional pedagogical practices.

The dissertation is divided into two parts. The first part examines …

Differential Item Functioning Of The Region-Based National Examinationequipment, Adi Setiawan, Gulzhaina Kuralbaevna Kassymova, Vianney Mbazumutima, Anggit Reviana Dewi Agustyani

REID (Research and Evaluation in Education)

This research aims to detect Differential Item Functioning (DIF) in the 2014/2015 National Examination Questions in mathematics of junior high schools and equivalent- level schools in the Yogyakarta region as a reference group and the South Kalimantan region as a focus group using the Likelihood Ratio Test (LRT) method, Area Measure Raju, and Lord. A sensitivity analysis was conducted to determine the most sensitive method. The data consisted of 5,465 National Examination papers of the students from the two regions who worked on type A questions. A sample of 1,000 exam papers for each region was established using the simple …

Exploring The Perceptions Of Literacy In Assessment For Learning Among High School English Teachers, Sukarno Sukarno, Nur Hidayanto Pancoro Setyo Putro, Ihtiara Fitrianingsih, Yasir Ayed Alsamiri, Fadhl Mohammed Awadh Gharamah, Ingrid A. Gavilan Tatin

REID (Research and Evaluation in Education)

In the field of English language education, the crucial role of assessment for learning (AfL) requires teachers to possess robust assessment literacy. This study explores AfL literacy among high school English teachers in Yogyakarta, Indonesia by utilizing Alonzo's validated AfL survey that establishes a comprehensive six-factor model, delineating teachers as assessors, pedagogists, student partners, motivators, learners, and stakeholder partners. Exploiting confirmatory factor analysis and examining demographic variations, this quantitative research invited 202 English teachers in the Special Region of Yogyakarta, Indonesia selected purposively based on the geographical service area. Data were collected through an online questionnaire adopting Alonzo's 42-questions AfL …

Understanding Barriers To Optimal Supervision And Delivery Of The Nationalcertificate (Vocational) Curriculum Through Tvet College Lecturers’Reflective Evaluations, Angelona Rewhydah Williams, Karel Prins, Bongani Innocent Nkambule, Sindile Amina Ngubane

REID (Research and Evaluation in Education)

Propounded by reflective theory, this qualitative case study drew on TVET lecturers' reflective evaluations of factors that they considered to have a bearing on optimal supervision and delivery of the National Certificate (Vocational) curriculum. Data were collected from participants across three campuses of a TVET college in the Eastern Cape Province, South Africa. Twelve lecturers of different seniority were purposively sampled and interviewed in two focus group sessions. The first focus session involved six participants: four post-level 2 and two post-level 3 personnel recognized by the South African Council for Educators (SACE) as "office-based lecturers" and classified within the middle …

Number Sense Profile Of Prospective Elementary School Teachers In Blendedmathematics Learning, Welly Novitasari, Herwin Herwin, Supartinah Supartinah, Putri Wulandari, Budiharti Budiharti

REID (Research and Evaluation in Education)

Number sense is a skill that contributes significantly to learning mathematics. However, number sense is often positioned as a fundamental skill whose development is more focused on children. The contribution of number sense in mathematics is even more apparent at higher levels of education. Ironically, number sense seems ignored and has become a rarely studied topic in higher education. Thus, the student’s number sense ability profile seems buried with various problems. This study aims to reveal the profile of prospective elementary school teachers’ number sense abilities and the factors causing their failure in solving math problems during the implementation of …

A Psychometric Evaluation Of An Item Bank For An English Readingcomprehension Tool Using Rasch Analysis, Louis Wai Keung Yim, Che Yee Lye, Poh Wee Koh

REID (Research and Evaluation in Education)

This study reports the psychometric evaluation of an item bank for an Assessment for Learning (AfL) tool to assess primary school students’ reading comprehension skills. A pool of 46 primary 1 to 6 reading passages and their accompanying 522 multiple choice and short answer items were developed based on the Progress in International Reading Literacy Study (PIRLS) assessment framework. They were field-tested at 27 schools in Singapore involving 9834 students aged between 7 and 13. Four main com- prehension processes outlined in PIRLS were assessed: focusing on and retrieving ex- plicitly stated information, making straightforward inferences, interpreting and inte- grating …

Simulation Of Low-High Method In Adaptive Testing, Rukli Rukli, Noor Azeam Atan

REID (Research and Evaluation in Education)

The era of disruption significantly engineered a classic testing system into an adaptive testing system where each test taker takes a unique test. However, the carrying capacity of the adaptive testing system engineering is experiencing obstacles in terms of the method of presenting the test questions. The study aims to introduce the low-high adaptive tracking method with the item response theory approach, where the difficulty level of the questions is adapted to the test takers' abilities. The number of test questions in the question bank is 400 questions. Data analysis used the Bilog-MG program. The range of the difficulty level …

Construction Of An Instrument For Evaluating The Teaching Process In Highereducation: Content And Construct Validity, Risky Setiawan, Wagiran Wagiran, Yasir Alsamiri

REID (Research and Evaluation in Education)

This study aims to reveal the content validity, construct validity, and reliability of the instrument for evaluating the teaching process in higher education. This research is development research applying the ADDIE model from Molenda. The indicators evaluated consist of context, inputs, processes, and products. The sample consisted of 1200 students from eight faculties, each represented by three study programs. Data analysis uses three stages: content validity test analysis using the V-Aiken method involving six panellists or experts; construct validity test using Confirmatory Factor Analysis (CFA). Quantitative descriptive analysis and interpretive qualitative analysis used the Miles and Huberman method. The results …

Constructing And Providing Content Validity Evidence Through The Aiken's Vindex Based On The Experts' Judgments Of The Instrument To Measuremathematical Problem-Solving Skills, Nia Kania, Yaya S. Kusumah, Jarnawi Afgani Dahlan, Elah Nurlaelah, Ferit Gürbüz, Ebenezer Bonyah

REID (Research and Evaluation in Education)

Test content-based proof of validity is a type of evidence that supports the validity of a measuring instrument. This research aims to develop a mathematical problem-solving assessment instrument utilizing five experts. This study is classified as developmental research and follows a research design that includes two separate stages: the preliminary design stage and the prototype stage. However, its application is restricted to Prototype 1 and Prototype 2, specifically for expert evaluation. This instrument was designed explicitly for grade VIII students studying mathematics, covering all the topics from the odd semesters. The analysis progressed through three distinct stages— curriculum analysis, content …

Stability Of Estimation Item Parameter In Irt Dichotomy Considering The Number Of Participants, Zulfa Safina Ibrahim, Heri Retnawati, Alfred Irambona, Beatriz Eugenia Orantes Pérez

REID (Research and Evaluation in Education)

This research is related to item response theory (IRT) which is needed to measure the goodness of a test set, while item parameter estimation is needed to determine the technical properties of a test item. Stability of item parameter estimation is conducted to determine the minimum sample that can be used to obtain good item parameter estimation results. The purpose of this study is to describe the effect of the number of test takers on the stability of item parameter estimation with the Bayes method (expected a posteriori, EAP) on dichotomous data. The minimum sample for stability is 1,000 participants …

Predictor Factors For Teacher Innovative Behavior During Curriculum Reform: The Role Of Cognitive Flexibility, Positive Affect, And Negative Affect, Khairunnisa Syafira Dumbi, Stephanie Yuanita Indrasari

REID (Research and Evaluation in Education)

Teacher innovative behavior is a key capability for maximizing teacher performance as well as the student learning process, especially when teachers face new changes in their education system such as curriculum reform. However, insufficient attention has been given to understanding the interaction of predictor factors to encourage teacher innovative behavior. This study aims to assess the role of cognitive flexibility, positive affect, and negative affect in predicting teacher innovative behavior. A cross-sectional and quantitative design was used for this study. The data collection procedure used convenience sampling, with questionnaires distributed online via social media through several teacher communities. Three instruments …

Effects Of Learning Languages On Volunteering, Ailina Makhmut Nursultankyzy, Natalya Nikolaevna Khan, Gulzhaina Kuralbaevna Kassymova, Ardak Kenesbayevna Kalimoldayeva, Elmira Kanatovna Ketebayeva

REID (Research and Evaluation in Education)

Learning languages is an important aspect of modern times that influences people's thinking which was studied by many researchers. Today the knowledge of different languages is becoming an increasingly significant aspect of volunteering, having a profound impact on its various aspects. This article discusses the importance of knowing foreign languages for developing empathy, improving communication skills, and expanding the worldview of volunteers using a questionnaire consisting of 12 questions divided into three main sections (empathy, communication skills, and worldview). Each section includes four questions related to the specific aspect that are given randomly. Responses were received from random respondents from …

Automatic Generation Of Physics Items With Large Language Models (Llms), Moses Oluoke Omopekunola, Elena Yu Kardanova

REID (Research and Evaluation in Education)

High-quality items are essential for producing reliable and valid assessments, offering valuable insights for decision-making processes. As the demand for items with strong psychometric properties increases for both summative and formative assessments, automatic item generation (AIG) has gained prominence. Research highlights the potential of large language models (LLMs) in the AIG process, noting the positive impact of generative AI tools like ChatGPT on educational assessments, recognized for their ability to generate various item types across different languages and subjects. This study fills a research gap by exploring how AI-generated items in secondary/high school physics aligned with educational taxonomy. It utilizes …

Developing A Pancasila Students' Character Instrument: Proof Of Construct Validity And Estimation Of Construct Reliability, Wiwin Mistiani, Edi Istiyono, Amir Syamsudin, Nor Hasnida Md. Ghazali

REID (Research and Evaluation in Education)

This study aims to prove construct validity and estimate construct reliability of the character scale of Pancasila students by using confirmatory factor analysis. The research approach used is quantitative with a sample of 200 high school students selected at random. This study measured the variables by six aspects, 15 indicators, and 20 questions. The six aspects include faith and piety to God Almighty, global nationality, cooperation, independence, critical reasoning, and creativity. The data analysis technique used is confirmatory factor analysis. The study results concluded that the instrument for assessing the character of Pancasila students in high school had a loading …

Changing Images Of Favorite Schools After The Student Admission System Reform In Indonesia, Janu Arlinwibowo, Sri Fajar Martono, Ikhya Ulumudin, Hermanto Waruwu, Yufridawati Yufridawati, Yunita Murdiyaningrum, Gupita Nadindra Fatima, Benjamin Leong Tack Khang

REID (Research and Evaluation in Education)

The study aims to unearth public perceptions of the label of favorite schools after the implementation of student admission reform in Indonesia, namely the zoning system, and to explore the underlying causes. Adopting a qualitative method with a phenomenological approach, this study involved participants from nine provincial education services, nine regency education services, 72 teachers, and 72 parents. Data collected through in-depth interviews and focus groups reveal three distinctive perceptions of favorite schools. The first view posits that after the implementation of the zoning system, the quality among schools is relatively similar, indicating that the new policy is effective in …

Transformation Of Accreditation Of Higher Education In Health In Indonesia: Qualitative Assessment, Yefta Moenadjat, Insan Sosiawan A. Tunru, Sutrisno Soemardjo

The New Ropanasuri Journal of Surgery

Accreditation of higher education in health in Indonesia shifts to the qualitative assessment driven to ensure quality improvement. The method replaces the former quantitative assessment method that has been applied since 2012; it will be effective in January 2025. The driving factors and rationale of transformation are outlined in this article. The rationale of the qualitative assessment that applies principle–based accreditation and the quantitative assessment that applies rule–based accreditation is reviewed here. Information on the accreditation procedure and the goal is presented, aiming for institutions to prepare for this new version of accreditation.

Integrating Rehabilitation Services Into Routine Care Of Rheumatoid Arthritis May Reduce The Inflammatory Response: A Hospital-Based Follow-Up Study In Taiwan, Hui-Ju Huang, Wei-Jen Chen, Hanoch Livneh, Hua-Lung Huang, Ming-Chi Lu, Tzung-Yi Tsai, Multiple Additional Authors

Counselor Education Faculty Publications and Presentations

Background and Objectives:

For persons with rheumatoid arthritis (RA), the accompanying systemic inflammatory conditions often insidiously damage extra-glandular organs, causing poor outcomes. Despite evidence manifesting the application of rehabilitation services (RSs), the association between RSs use and changes in the inflammatory response among persons with RA has not yet been established. With that in mind, this study aimed to evaluate changes in C-reactive protein (CRP) and erythrocyte sedimentation rate (ESR) before and after long-term RSs use.

Materials and Methods:

For this two-group pre and posttest study, medical data of 4505 persons with RA aged 20–70 years between 2012 and 2017 …

The Importance Of School Leadership Practices In Social-Emotional Learning Of Elementary And Secondary Schools, Janice Filer Ph.D.

The Scholarship Without Borders Journal

Abstract: The purpose of this qualitative phenomenological study was to interview school administrators in the Los Angeles area who utilized social-emotional learning programs in their individual schools. Since the worldwide pandemic, schools have emphasized the importance of social-emotional learning. This research was conducted in an effort to provide school administrators with additional strategies to support the importance of social-emotional learning. In-depth interviews were conducted to collect leadership practices. The data was collected during the Spring 2024 and Summer 2024 school calendar year. Seven participants completed the interview process. Four themes emerged from the data including the importance of equity, belonging/relationships, …

A Rank-Order Alternative For Non-Parametric Analysis With The General Linear Model, Jeffry White

Journal of Educational Research and Practice

Violations of normality and homogeneity are common in educational data. When this occurs, the use of parametric statistics may be inappropriate. A generalized form of nonparametric analyses based on the Puri and Sen L statistic provides an alternative approach. Using a chi-square distribution, this technique is easy to apply and has significant power. Another advantage of the L statistic is its utility to link nonparametric tests with their parametric counterparts. After rank-ordering and analyzing the data, an adjustment is made by calculating L instead of relying on the parametric test statistic. This permits the researcher to choose between parallel strategies …

Reputation Management: Branding A Higher Education Graduate Program To Increase Visibility, Vanderbilt Artaxerxes Brabson Iv

Student Theses and Dissertations

This capstone project offers a comprehensive training tool for non-communication professionals working in higher education programs. Its main goals are to give users (1) a summary of valuable research on higher education reputation management, branding, and strategic communication, and (2) an online toolkit for non-communication professionals to boost the visibility and impact of their educational initiatives. By providing practical, actionable resources, these tools help higher education professionals effectively handle the challenges of strategic communication management. The toolkit is especially useful because it delivers evidence-based strategies and methods that can be directly applied to improve communication efforts in higher education. It …

Reputation Management: Branding A Higher Education Graduate Program To Increase Visibility, Vanderbilt Artaxerxes Brabson Iv

Student Theses and Dissertations

This capstone project provides a comprehensive training tool for non-communication professionals working within higher education programs. Its primary objectives are to provide users (1) a summary of valuable research on higher education reputation management, branding, and strategic communication, as well as (2) an online toolkit for non-communication professionals to enhance the visibility and impact of their educational initiatives. By providing practical, actionable deliverables, these resources can enable higher education professionals to effectively navigate the complexities of strategic communication management. The toolkit is particularly valuable as it offers evidence-based strategies and methodologies that can be directly applied to improve the communication …

Society Of American Foresters (Saf) Academic Publications As They Relate To The Curriculum Standards For Saf Forestry Program Accreditation: A Content Analysis, Lisa M. Bentley

Electronic Theses and Dissertations

Goals that impact decision-making components within academic programs in the university system have the potential to define and change the direction of a field of study, and, with that, should be thoughtfully approached with a certainty of evidence that supports responsible paradigms in forestry education. Periodic revision of the standards for accreditation by SAF influences institutional expectations that result in shifts in forestry education and should be consistent with professional forestry standards. The methodology utilized in this research contextualizes data from a sociological perspective intentionally as a method to holistically integrate data as evidence to support the interpretation of Society …

Exploring Teacher Pathway Programs: The Importance Of High School Students’ Perspectives, Mackenzie C. Kasgorgis

Honors College Theses

This thesis explores and investigates the effectiveness of teacher pathway programs for high school students interested in education. The aim is to find and discuss the importance of teacher pathway programs and how they benefit the participants involved. This paper intertwines my analysis of the extant scholarly literature and my use of qualitative research methods during my immersion in two summer programs. Through participant observations and a survey, I gained insights into the participants’ perspectives that provided me with raw, unbiased views and opinions on the teacher pathway program. Student participants’ perspectives include the experiences they faced during the program, …

An Investigation Into Two Mathematics Score Reports: Problem Solving Measure (Psm) And Measurement Of Academic Progress (Map), Benjamin Lawson, Jonathan D. Bostic

Mid-Western Educational Researcher

K-12 teachers and staff need high-quality score reports that accurately report students’ results. However, research about score reports that K-12 teachers and staff use indicates that they infrequently are given opportunities to share feedback to developers. To address this, we gathered feedback from teachers and administrators, in an explanatory sequential design of a mixed-methods study. A purpose of the study was to examine score report quality from two mathematics assessments. Two researchers surveyed in-service professionals (n = 34) about two score reports. Then, we conducted follow-up interviews (n = 6) to further unpack what school personnel find valuable and useful …

Connecting Through Screens: A Case Study On Student Engagement In Online Esol Classes In The Post Pandemic Era, Meral Kacar

Master's Theses

The sudden shift to emergency online education during the global pandemic affected various academic fields, including English language teaching. While many schools faced challenges, some successfully implemented online educational programs. Despite the large impact of the global pandemic on education, there is limited research that examines the effectiveness of online ESOL programs with a focus on the key factors that contribute to student engagement. This thesis explores the factors contributing to effective online ESOL instruction, focusing on fostering engagement and improving teaching practices in the post-pandemic era. The study for this thesis employs an instrumental case study methodology, focusing on …

Comparison Of Machine Learning Packages In R: An Application With Open Dataset, Daniel O. Oyeniran

Chinese/English Journal of Educational Measurement and Evaluation | 教育测量与评估双语期刊

Machine learning has become a prominent approach for prediction in many fields of endeavor, including education. Python and R have become leading software while different packages have been developed for the purpose of carrying out machine learning analysis with differences in functions in R. This paper explored the functional and empirical differences of two major machine learning packages in R to estimate metrics. This comparative study provides a comprehensive guide for leveraging machine learning packages in R, facilitating informed decision-making, and advancing the adoption of effective methodologies in data-driven research and application. Findings in this study ensure that researchers make …

Detecting Compromised Items In Computerized Linear Testing: A Novel Approach Using Autoencoders And Bert, Yiqin Pan, Sandip Sinharay

Chinese/English Journal of Educational Measurement and Evaluation | 教育测量与评估双语期刊

Item compromise is a significant concern in the testing industry. We propose a novel approach for detecting compromised items in computerized linear testing by analyzing response times and scores. Our method consists of three primary components. First, we employ autoencoders, a type of neural network, to model response times and scores, capturing the unique characteristics of each item and examinee. Second, we develop a BERT-based model to analyze examinees’ response patterns, predict expected response times. The characteristic vectors generated by the autoencoders are integrated into the BERT-based model, enhancing its ability to recognize response patterns. Third, an algorithm is designed …

Automated Text Scoring In The Age Of Generative Ai For The Gpu-Poor, Chris M. Ormerod, Alexander Kwako

Chinese/English Journal of Educational Measurement and Evaluation | 教育测量与评估双语期刊

Abstract Current research on generative language models (GLMs) for automated text scoring (ATS) has focused almost exclusively on querying proprietary models via Application Programming Interfaces (APIs). Yet such practices raise issues around transparency and security, and these methods offer little in the way of efficiency or customizability. With the recent proliferation of smaller, open-source models, there is the option to explore GLMs with computers equipped with modest, consumer-grade hardware—that is, for the “GPU poor.” In this study, we analyze the performance and efficiency of open-source, small-scale GLMs for ATS. Results show that GLMs can be fine-tuned to achieve adequate, though …

A Predictive Correlational Study Of The Relationship Between Two-Year College Faculty Participation In Professional Development And Career Commitment, Faculty Leadership, And Professional Service Activities, Deborah Josey Burks

Doctoral Dissertations and Projects

The purpose of this predictive correlational non-experimental study was to examine the relationships between faculty participation in professional development, and career commitment, faculty leadership, and engagement in professional service activities. The predictor variable was faculty professional development and criterion variables were career commitment, faculty leadership, and engagement in professional service by faculty employed within two-year colleges in the southeastern United States. This study contained a convenience sample of 76 faculty employed at two-year colleges within a system of colleges. Data was gathered utilizing the combined self-reported Career Growth Opportunity Measure (Adams, 1999), the Teacher Leadership Inventory [TLI] (Angelle & DeHart, …