Education Commons^™

Cognitive Load Scale In Learning Formal Definition Of Limit: A Rasch Model Approach, Rina Oktaviyanthi, Ria Noviana Agus, Mark Lester B. Garcia, Kornkanok Lertdechapat

Mathematics Faculty Publications

Constructing proofs for the limit using the formal definition induces a high cognitive load. Common assessment tools, like cognitive load scales, lack specificity for the concept of limits. This research aims to validate an instrument tailored to assess cognitive load in students focused on the formal definition of limits, addressing the need for diverse strategies in education. The research employs a quantitative survey design with a Rasch model approach, utilizing a data collection instrument in the form of a questionnaire. Subsequently, the data are analyzed by focusing on three aspects: (1) item fit to the Rasch model, (2) unidimensionality, and …

Item Analysis Of Reading Comprehension Questions For English Proficiency Test Using Rasch Model, Henda Harmantia Dewi, Siti Maftuhah Damio, Sukarno Sukarno

REID (Research and Evaluation in Education)

The need to take English as a foreign language proficiency test (known as TOEFL [Test of English Language Proficiency]) has been gaining popularity in Indonesia. The increasing demands for such a test and its expensive cost have reinforced many institutions to develop TOEFL instruments and administer the test internally. However, constructing a test instrument is a complex process that makes conducting item analysis become more challenging. Meanwhile, item analysis is crucial to assess the items' quality. Therefore, this study reported the results of statistically analyzing 20 questions of TOEFL reading comprehension that were analyzed in terms of the test reliability, …

Score Equating Between Aeps-2 And Aeps-3 For 0-3 Year Olds, Yuyan Xia

Theses and Dissertations--Education Sciences

Over the past two decades, the emphasis on educational equity in early childhood education (ECE) and early childhood special education (ECSE) has highlighted the importance of assessment through policies and regulations. Ensuring accurate assessment scores is a fundamental aspect of this trend. The release of the Assessment, Evaluation, and Programming System for Infants and Children, Third Edition (AEPS-3) in December 2021 led to a shift from the Second Edition (AEPS-2) in child development scoring. In order to harmonize the previous and updated assessment versions for children aged 0-3 across six developmental domains, a common item non-equivalent design, featuring fixed parameter …

Analysis Of Middle Semester Exam Subjects For Automotive Vehicle Chassis (Avc) To Improve The Implementation Management Of Learning Evaluation, Susi Tri Umaroh, Soeryanto Soeryanto, Warju Warju, Marniati Marniati

Jurnal Pendidikan Vokasi

Entering the 21st century with the industrial revolution 4.0, the speed of progress in information and communication technology has become unstoppable. One of the most benefited from these advances in the world of education. Exploration of learning resources, and discussion forums, for the evaluation of learning can be done easily because of technological advances. Learning evaluation can use a digital platform, and it is very easy to analyze to improve the management of the implementation of learning evaluation. This study was conducted to analyze the mid-semester test instrument, which includes Higher Order Thinking Skills (HOTS) questions and is used to …

Development And Standardization Of A Rating Scale Designed For Floorball Skills Diagnostics Of Young School-Age Children, Zuzanna Dragounova

Baltic Journal of Health and Physical Activity

Background:‪The purpose of the study is to develop a standardized diagnostic tool designed to predict the level of the tested floorball skills in young school-age children that is necessary for future game performance.Material and methods:‪For the construction of the Guttman-type scale, the Rasch model was applied. The methodology employed the procedures for standardization by Stochl & Musalek, fit functions to determine the fit of the data model, KR-20 coefficient for the reliability calculation, Fleiss’ kappa coefficient to determine the inter-rater agreement, and PCA of residuals to determine the unidimensionality.Results:‪Only 9 items out of a total of 30 were selected and …

Impacts Of Model Misspecification On Individual-Level Differential Item Functioning Detection In Multilevel Data Using Hierarchical Generalized Linear Model, Yue Yin

USF Tampa Graduate Theses and Dissertations

Hierarchical generalized linear modeling (HGLM) has become more and more popular in detecting differential item functioning (DIF) items in multilevel contexts due to its flexibility and efficiency. This dissertation aims to investigate and compare four different HGLM DIF detection models under various DIF conditions with multilevel data. Three studies were conducted in this dissertation. The first two studies were simulation studies and the third one was an empirical study. Simulation Study One compared the performance of two-level HGLM and three-level HGLM with the individual-level covariate (HGLM-IL) on detecting individual-level DIF items with two-level and three-level data structures. Simulation Study Two …

Beliefs About Teaching (Bats2) - Rasch Model Analysis Of An Instrument Based On Intasc Critical Dispositions, W. Steve Lang, Lasonya Moore, Judy R. Wilkerson

University of South Florida (USF) M3 Publishing

This research is one of a chain of studies on the development and testing of a battery of teacher dispositions assessments designed to measure the InTASC critical dispositions (InTASC, 2013). Accredited programs in the US are required to use these standards and to assess teacher dispositions. Here, an alternate form of the Thurstone agreement scale (Beliefs About Teaching Survey or BATS, version 2, Form B) was validated with US undergraduate teacher education majors from two public universities (N=149) with this population. Results indicate that Form B of BATSv2 has comparable results to Form A and the first version of the …

Rise And Shine: A Comparison Of Item Fit Statistics For The Rasch Model, Glenn T. Waterbury

Dissertations, 2020-current

The Rasch model implies that the relation between examinee ability and the probability of correctly answering an item can be defined solely by a small set of parameters. In the case of Rasch modeling, there are only two parameters: the ability of an examinee and the difficulty of an item. When the data meet the requirements of the Rasch model, it possesses several appealing properties that distinguish it from Classical Test Theory and more complex Item Response Theory models.

However, the desirable properties of the Rasch model only exist when the data meet its strict requirements. Therefore, it is vital …

An Analysis Of Javanese Language Test Characteristic Using The Rasch Model In R Program, Muchlisin Muchlisin, Djemari Mardapi, Farida Agus Setiawati

REID (Research and Evaluation in Education)

One skill required to solve a problem in the 21st century is communication. Two international languages that are important in communication and thought at school are English and German language. However, beside international language, the local language, such as the Javanese language, is also essential and need to be maintained. The purpose of this study is to analyze the Javanese language test characteristics. This study was explorative research with secondary data collected by documentation of 220 students responses to the 50 multiple choice item of Javanese language test in the 11th grade of vocational high school. Data were analyzed using …

Application Rasch Model Using R Program In Analyze The Characteristics Of Chemical Items, Untung Desy Purnamasari, Badrun Kartowagiran

Jurnal Inovasi Pendidikan IPA

Chemistry is one of the subjects taught in high school. To find out and assess students' understanding regarding chemistry subjects in one semester can be proven by a test. The tests used must have good quality. This study aims to provide information about the characteristics of chemical items test using the Rasch model. Descriptive explorative was used in this study. The subject of the study were tenth grade students in Xaverius Senior High School taken the final semester examination on chemistry subject. The object of this research were the form of item tests and student answer sheets. Data collection techniques …

A Construct Map For Tpack: Developing An Empirically Derived Description Of Increasing Tpack Proficiency, Ralph Saubern, Daniel Urbach, Matthew Koehler, Michael Phillips

Ralph Saubern

Analisis Iteman Dan Model Rasch Pada Pengembangan Instrumen Kemampuan Berpikir Kritis Peserta Didik Sekolah Menengah Kejuruan, Sri Nuryanti, Muhammad Masykuri, E. Susilowati

Jurnal Inovasi Pendidikan IPA

Tujuan penelitian ini menganalisis instrumen kemampuan berpikir kritis. Penelitian ini merupakan penelitian kuantitatif. Data diperoleh dengan mengembangkan instrumen penilaian kemampuan berpikir kritis. Responden sebanyak 90 peserta didik yang diambil dari 3 SMK yang ada di kota Surakarta. Analisis data menggunakan perangkat lunak (software) analisis Iteman versi 3.0 dan Winstep versi 3.73. Metode yang digunakan dalam analisis ini yaitu metode tes klasik yaitu Iteman dan pemodelan Rasch. Hasil analisis dengan pemodelan Iteman diperoleh nilai Alpha Cronbach 0,80 berarti reliabilitas dari soal ini tinggi. Sedangkan dengan model Rasch Alpha Cronbach 0,74 berarti reliabilitas antara person dan item, sedangkan …

Reduce Randomly Guessing Effects In A University Generic Skills Test, Luc T. Le

Dr Luc Tu Le

Improving Measurement Of Ambiguity Tolerance Among Teacher Candidates, Kent Allan Rittschof

Georgia Educational Research Association Conference

The process of learning often requires dealing with the ambiguity of uncertain interpretations. A learner’s tolerance for ambiguity involves the degree of willingness to accept or adapt to unfamiliar, unpredictable, or uncertain situations and ideas. This study examined the measurement of ambiguity tolerance (AT) among teacher candidates. Pre-service teachers (n = 114) attending a medium size university in the southeastern United States were administered McLain's (1993) Multiple Stimulus Types Ambiguity Tolerance (MSTAT-I) scale. Analyses were first conducted on item responses from the MSTAT-I, then on item responses from the MSTAT-II (McLain, 2009), an instrument comprised of a subset of 13 …

How Success In A Task Depends On The Skills Level: Two Uncertainty-Based Justifications Of A Semi-Heuristic Rasch Model, Joe Lorkowski, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

The more skills a student acquires, the more successful this student is with the corresponding tasks. Empirical data shows that the success in a task grows as a logistic function of skills; this dependence is known as the Rasch model. In this paper, we provide two uncertainty-based justifications for this model: the first justification provides a simple fuzzy-based intuitive explanation for this model, while the second -- more complex one -- explains the exact quantitative behavior of the corresponding dependence.

National Assessment Program : Ict Literacy 2008 : Year 6 And Year 10 : Technical Report, Chris Freeman, John Ainley, Julian Fraillon, Eveline Gebhardt, Kate O'Malley, Wolfram Schulz

Dr Wolfram Schulz

This report describes the technical aspects of the National ICT Literacy Sample Assessment and summarises the main activities involved in the data collection, the data collection instruments and the analysis and reporting of the data. Chapter 2 summarises the development of the assessment domain and describes the process of item development and construction of the instruments. Chapter 3 reviews the sample design and describes the sampling process. Chapter 3 also describes the weighting procedures that were implemented to derive population estimates. Chapter 4 summarises the field administration and data management procedures, including quality control and the cleaning and coding of …

Evaluation Of Item Parameter Recovery Estimation By Acer Conquest Software, Luc T. Le, Ray Adams

Dr Luc Tu Le

ACER ConQuest (Adams, Wu, and Wilson, 2012) has been popularly used for analysing testing and assessment data. Two of the most common estimation methods for Rasch measurement models (Rasch, 1960/1980) are available in this software, marginal maximum likelihood estimation (MML) and joint maximum likelihood estimation (JML). This study is concerned with item parameter recovery for the dichotomous Rasch model. Our primary focus is on comparing JML and MML when the assumptions of MML are violated, that is the abilities are not sampled from the distribution that is assumed in the estimation.

Evaluation Of Item Parameter Recovery Estimation By Acer Conquest Software, Luc T. Le, Ray Adams

Professor Ray Adams

ACER ConQuest (Adams, Wu, and Wilson, 2012) has been popularly used for analysing testing and assessment data. Two of the most common estimation methods for Rasch measurement models (Rasch, 1960/1980) are available in this software, marginal maximum likelihood estimation (MML) and joint maximum likelihood estimation (JML). This study is concerned with item parameter recovery for the dichotomous Rasch model. Our primary focus is on comparing JML and MML when the assumptions of MML are violated, that is the abilities are not sampled from the distribution that is assumed in the estimation.

Effects Of Item-Level Feedback On The Ratings Provided By Judges In A Modified-Angoff Standard Setting Study, Michael R. Peabody

Theses and Dissertations--Education Sciences

Setting performance standards is a judgmental process involving human opinions and values as well as technical and empirical considerations and although all cut score decisions are by nature arbitrary, they should not be capricious. Establishing a minimum passing standard is the technical expression of a policy decision and the information gained through standard setting studies inform these policy decisions. To this end, it is necessary to conduct robust examinations of methods and techniques commonly applied to standard setting studies in order to better understand issues that may influence policy decisions.

The modified-Angoff method remains one of the most popular methods …

National Assessment Program : Ict Literacy 2008 : Year 6 And Year 10 : Technical Report, Chris Freeman, John Ainley, Julian Fraillon, Eveline Gebhardt, Kate O'Malley, Wolfram Schulz

Julian Fraillon

This report describes the technical aspects of the National ICT Literacy Sample Assessment and summarises the main activities involved in the data collection, the data collection instruments and the analysis and reporting of the data. Chapter 2 summarises the development of the assessment domain and describes the process of item development and construction of the instruments. Chapter 3 reviews the sample design and describes the sampling process. Chapter 3 also describes the weighting procedures that were implemented to derive population estimates. Chapter 4 summarises the field administration and data management procedures, including quality control and the cleaning and coding of …

National Assessment Program : Ict Literacy 2008 : Year 6 And Year 10 : Technical Report, Chris Freeman, John Ainley, Julian Fraillon, Eveline Gebhardt, Kate O'Malley, Wolfram Schulz

Dr John Ainley

This report describes the technical aspects of the National ICT Literacy Sample Assessment and summarises the main activities involved in the data collection, the data collection instruments and the analysis and reporting of the data. Chapter 2 summarises the development of the assessment domain and describes the process of item development and construction of the instruments. Chapter 3 reviews the sample design and describes the sampling process. Chapter 3 also describes the weighting procedures that were implemented to derive population estimates. Chapter 4 summarises the field administration and data management procedures, including quality control and the cleaning and coding of …

Extended Mantel-Haenszel Procedure For Dif Detection : A Note On Its Implementation In Acer Conquest, Xiaoxun Sun

Xiaoxun Sun

Many methods have been developed for studying Differential Item Functioning (DIF). An approach developed by Holland and Thayer (1988) adapted the Mantel-Haenszel (MH) statistic (Mantel and Haenszel 1959). The author of this chapter has implemented MH procedure and its extension in an experimental version of ACER ConQuest which can handle DIF analysis for multiple focal groups and polytomous items. This chapter compares the extended MH procedure in DIF detection to the standardized item difficulty difference method based on the Rasch estimates for both dichotomous and polytomous items. The use of MH method in DIF analysis with multiple focal groups is …

National Assessment Program : Ict Literacy Technical Report, Eveline Gebhardt, Julian Fraillon, Wolfram Schulz, Kate O'Malley, Chris Freeman, Martin Murphy, John Lindsey

Martin Murphy

This report is concerned with procedures, processes and technical aspects of the National Assessment Program - ICT Literacy 2011 Literacy (NAP - ICTL 2011) and should be read in conjunction with the Public Report from Literacy NAP - ICTL 2011 which focuses on results and interpretation of results from that assessment. The first cycle of the NAP - ICTL was held in 2005 and provided the baseline against which future performance would be compared. The second cycle of the program was conducted in 2008 and was the first cycle where trends in performance were able to be examined.

Investigating Item Difficulty Change By Item Positions Under The Rasch Model, Luc T. Le, Van Nguyen

Dr Van Nguyen

Investigating Item Difficulty Change By Item Positions Under The Rasch Model, Luc T. Le, Van Nguyen

Dr Luc Tu Le

Quality Of Supervision Of Ph.D. Program Among Public Universities In Malaysia: A Rasch Model Analysis, Siti Aishah Hassan Ph.D.

Siti Aishah Hassan Ph.D.

This study examines Ph.D. students’ satisfaction with the supervision process at four selected universities in Malaysia. In addition, the study also investigated the psychometric properties of Quality Supervision Scale (QSS); specifically the scale dimensionality, construct validity, endorsibility, and estimation of item and person score reliability of the scales. The participants were 153 Ph.D. students of these universities. The QSS includes many qualities of effective supervision such as supervisor academic competency, research methods competency, attitude towards supervisee, faculty academic and moral supports and supervisees’ personal traits was distributed to the respondents. The Rasch model analysis was employed to analyze the data …

Multiple-Choice Questions Compared To Short-Answer Response : Which Assesses Understanding Of Chemistry More Effectively?, Ross Hudson

Dr Ross Hudson

The research inquires into the effectiveness of the two predominant forms of questions that are used on the Victorian Certificate of Education (VCE) Victoria Chemistry examination. These are multiple- choice questions and short-answer questions. This research examines not only the style of chemistry question but also the content type examined (recall and application questions) along with gender differences in students' responses to such questions. The research involved three phases, i) analysis of five years results from the VCE Chemistry examinations, ii) class trial testing students of both genders with structured questions that examined the same material content with each type …

The Analysis Of Measurement Equivalence In International Studies Using The Rasch Model, Wolfram Schulz, Julian Fraillon

Civics and Citizenship Assessment

No abstract provided.

Ameliorating Culturally Based Extreme Response Tendencies To Attitude Items, Maurice Walker

Maurice Walker

No abstract provided.

The Influence Of Equating Methodology On Reported Trends In Pisa, Eveline Gebhardt, Ray Adams

Prof Ray Adams

In 2005 PISA published trend indicators that compared the results of PISA 2000 and PISA 2003. This paper explores the extent to which the outcomes of these trend analyses are sensitive to the choice of test equating methodologies, the choice of regression models and the choice of linking items. To establish trends, PISA equated its 2000 and 2003 tests using a methodology based on Rasch Modelling that involved estimating linear transformations that mapped 2003 Rasch-scaled scores to the previously established PISA 2000 Rasch-scaled scores. This paper compares the outcomes of this approach with an alternative, which involves the joint Rasch …