Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 5 of 5
Full-Text Articles in Education
The Influence Of Equating Methodology On Reported Trends In Pisa, Eveline Gebhardt, Ray Adams
The Influence Of Equating Methodology On Reported Trends In Pisa, Eveline Gebhardt, Ray Adams
Prof Ray Adams
In 2005 PISA published trend indicators that compared the results of PISA 2000 and PISA 2003. This paper explores the extent to which the outcomes of these trend analyses are sensitive to the choice of test equating methodologies, the choice of regression models and the choice of linking items. To establish trends, PISA equated its 2000 and 2003 tests using a methodology based on Rasch Modelling that involved estimating linear transformations that mapped 2003 Rasch-scaled scores to the previously established PISA 2000 Rasch-scaled scores. This paper compares the outcomes of this approach with an alternative, which involves the joint Rasch …
Reliability As A Measurement Design Effect, Ray Adams
Reliability As A Measurement Design Effect, Ray Adams
Prof Ray Adams
Test reliability is a concept central to classical test theory and it is commonly stated as a requirement that a test attain a certain level of reliability before it be considered of sufficient quality for practical use. This article discusses the role of reliability in item response theory, and in particular the role of reliability in contexts where matrix sampling designs are used and concern is with the estimation of population parameters rather than the measurement of individuals. The concept of a measurement design effect is introduced. This concept parallels the concept of sampling design effects, in that it describes …
Response To 'Cautions On Oecd's Recent Educational Survey (Pisa), Ray Adams
Response To 'Cautions On Oecd's Recent Educational Survey (Pisa), Ray Adams
Prof Ray Adams
This paper refers to the work of S J Prais who questioned the outcomes of the Organisation for Economic Cooperation and Development's PISA survey of the reading, mathematics and science attainments of fifteen year-olds. Prais suggested that methodological flaws in Programme for International Student Assessment (PISA) had resulted in an apparent improvement in the attainment of British students, particularly when compared to their Swiss and German counterparts. This paper responds to Prais's criticisms, noting that when Prais's conjectures are tested with empirical data they are not supported. The author argues that many of Prais's criticisms are due to an incomplete …
The Implications Of Halo Effects And Item Dependencies For Objective Measurement, T Mcnamara, Ray Adams
The Implications Of Halo Effects And Item Dependencies For Objective Measurement, T Mcnamara, Ray Adams
Prof Ray Adams
Analysis of data from performance-based language tests presents the problem of dealing appropriately with bundles of items that are linked to particular tasks. The authors use recent developments in Rasch modelling which allow analysis of bundles of items (or sets of ratings) to examine the extent of dependence between items and ratings, and they explore the implications of those dependences on the assessments. Data from two performance assessments are used in two studies of this problem.
Broadening Assessment, Improving Fairness? Some Examples From School Science, Jan Lokan, Ray Adams, Brian Doig
Broadening Assessment, Improving Fairness? Some Examples From School Science, Jan Lokan, Ray Adams, Brian Doig
Prof Ray Adams
Following a review of equity issues in testing, especially in relation to science, two assessment programmes in school science, each of which was designed both to probe a range of aspects of student performance and for use in a large-scale study, are discussed. Both have been developed and used in the 1990s, one is Australia only and the other in many countries. The programmes and the range of aspects they attempt to assess are described and results are cited for several groups of students. Issues of fairness through broadening of large-scale assessments to include tasks requiring students to construct their …