Open Access. Powered by Scholars. Published by Universities.®

Education Commons

Open Access. Powered by Scholars. Published by Universities.®

Articles 1 - 28 of 28

Full-Text Articles in Education

A Learning Assessment Program Designed To Meet The Post-2015 Development Goals, Juliette Mendelovits, Maurice Walker, Ray Adams Oct 2015

A Learning Assessment Program Designed To Meet The Post-2015 Development Goals, Juliette Mendelovits, Maurice Walker, Ray Adams

Prof Ray Adams

No abstract provided.


Effects Of Student Reading Proficiency On Pisa Mathematics Items – A Dif Study Across Countries, Ray Adams, Luc Le Aug 2013

Effects Of Student Reading Proficiency On Pisa Mathematics Items – A Dif Study Across Countries, Ray Adams, Luc Le

Prof Ray Adams

No abstract provided.


Computer Adaptive Testing : A Feasibility Study, Siek Khoo, Geoff Masters, Ray Adams Aug 2013

Computer Adaptive Testing : A Feasibility Study, Siek Khoo, Geoff Masters, Ray Adams

Prof Ray Adams

The Australian National Office of Overseas Skills Recognition (NOOSR) commissioned ACER to investigate the feasibility of implementing computer adaptive testing (CAT) in NOOSR’s screening examinations for overseas trained professionals. The National Office administers screening examinations in seven professions (Dentistry, Dietetics, Occupational Therapy, Podiatry, Pharmacy, Physiotherapy and Veterinary Science) at venues throughout Australia and in up to fifty centres around the world. Surveys were conducted to gain an overview of the current methods and procedures used by NOOSR in the screening examinations. Issues related to the application of CAT to NOOSR’s screening examinations and the possible improvement of NOOSR ...


Displaying Test Analyses With Titan And Bigscale, Siek Khoo, Ray Adams, J Linacre Aug 2013

Displaying Test Analyses With Titan And Bigscale, Siek Khoo, Ray Adams, J Linacre

Prof Ray Adams

No abstract provided.


Quest: The Interactive Test Analysis System, Ray Adams, Siek Khoo Aug 2013

Quest: The Interactive Test Analysis System, Ray Adams, Siek Khoo

Prof Ray Adams

This is a guide to using Quest. Quest offers a comprehensive test and questionnaire analysis environment by providing a data analyst with access to the most recent developments in Rasch measurement theory, as well as a range of traditional analysis procedures. It includes an easy to use control language with flexible and informative output. Quest can be used to construct and validate variables based on both dichotomous and polychotomous observations. It scores and analyses such instruments as multiple choice tests, Likert type rating scales, short answer items, and partial credit items.


The Programme For International Student Assessment: An Overview, Ross Turner, Ray Adams Aug 2013

The Programme For International Student Assessment: An Overview, Ross Turner, Ray Adams

Prof Ray Adams

This paper provides an overview of the Programme for International Student Assessment (PISA), an on-going international comparative survey study of educational outcomes at age 15. PISA is sponsored by the Organization for Economic Co-operation and Development (OECD) and for the period 1998-2010 has been designed and implemented by a consortium led by the Australian Council for Educational Research (ACER).


Pisa : Frequently Answered Criticisms, Ray Adams Dec 2008

Pisa : Frequently Answered Criticisms, Ray Adams

Prof Ray Adams

Studies such as PISA that attempt to compare outcomes across educational systems are expensive and difficult to implement. Further, the results of such studies are routinely criticized by educational commentators - particularly when the results are not consistent with their preconceived ideas about the relative merits and efficiencies of various educational practices and systems. This chapter discusses what is done to ameliorate the threats to the validity of PISA in five areas often targeted by commentators and reviewers as sources of invalidity in international comparisons. The five areas discussed are: (1) sampling - are the samples of students who undertake assessment equally ...


The Influence Of Equating Methodology On Reported Trends In Pisa, Eveline Gebhardt, Ray Adams Dec 2006

The Influence Of Equating Methodology On Reported Trends In Pisa, Eveline Gebhardt, Ray Adams

Prof Ray Adams

In 2005 PISA published trend indicators that compared the results of PISA 2000 and PISA 2003. This paper explores the extent to which the outcomes of these trend analyses are sensitive to the choice of test equating methodologies, the choice of regression models and the choice of linking items. To establish trends, PISA equated its 2000 and 2003 tests using a methodology based on Rasch Modelling that involved estimating linear transformations that mapped 2003 Rasch-scaled scores to the previously established PISA 2000 Rasch-scaled scores. This paper compares the outcomes of this approach with an alternative, which involves the joint Rasch ...


The Impact Of Differential Investment Of Student Effort On The Outcomes Of International Studies, J Butler, Ray Adams Dec 2006

The Impact Of Differential Investment Of Student Effort On The Outcomes Of International Studies, J Butler, Ray Adams

Prof Ray Adams

International comparative assessments of student achievement, such as Trends in Mathematics and Science (TIMSS) and Programme for International Student Achievement (PISA) are becoming increasingly important in the development of evidence-based education policy. The potentially far-reaching influence of such studies underscores the need for these assessments to be valid and reliable. In education, increasing recognition is being given to motivational factors which impact on student learning. This research considers a possible threat to the validity of such studies by investigating the influence the amount of effort invested by test-takers has on their outcomes. Reassuringly, it is found that the reported expenditure ...


Application Of Multivariate Rasch Models In International Large-Scale Educational Assessments, Ray Adams, Margaret Wu, C Carstensen Dec 2005

Application Of Multivariate Rasch Models In International Large-Scale Educational Assessments, Ray Adams, Margaret Wu, C Carstensen

Prof Ray Adams

In large-scale educational assessments, such as the Programme for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS), a primary concern is with the estimation of the population-level characteristics of a number of latent variables and the relationships between latent variables and other variables. Typically these studies are undertaken in contexts in which there are constraints on sample size and individual student response time, yet there are high expectations with regard to the breadth of content coverage. These demands and constraints have resulted in such studies using rotated-booklet designs, with each student responding to a ...


The Mixed-Coefficients Multinomial Logit Model : A Generalized Form Of The Rasch Model, Ray Adams, Margaret Wu Dec 2005

The Mixed-Coefficients Multinomial Logit Model : A Generalized Form Of The Rasch Model, Ray Adams, Margaret Wu

Prof Ray Adams

Since Rasch's introduction of his item response models, there has been a proliferation of extensions and alternatives, each of which has a different name and different matching software package. The proliferation of models has, in some ways, been a hindrance to practitioners. This paper presents a generalised item response model that provides a unifying framework for a large class of Rasch-type models. The advantages of a single framework include mathematical elegance, generality in a single software package, and a facilitation of the development, testing, and comparison of new models. The unified model is a multidimensional item response model, the ...


Reliability As A Measurement Design Effect, Ray Adams Dec 2004

Reliability As A Measurement Design Effect, Ray Adams

Prof Ray Adams

Test reliability is a concept central to classical test theory and it is commonly stated as a requirement that a test attain a certain level of reliability before it be considered of sufficient quality for practical use. This article discusses the role of reliability in item response theory, and in particular the role of reliability in contexts where matrix sampling designs are used and concern is with the estimation of population parameters rather than the measurement of individuals. The concept of a measurement design effect is introduced. This concept parallels the concept of sampling design effects, in that it describes ...


Response To 'Cautions On Oecd's Recent Educational Survey (Pisa), Ray Adams Aug 2003

Response To 'Cautions On Oecd's Recent Educational Survey (Pisa), Ray Adams

Prof Ray Adams

This paper refers to the work of S J Prais who questioned the outcomes of the Organisation for Economic Cooperation and Development's PISA survey of the reading, mathematics and science attainments of fifteen year-olds. Prais suggested that methodological flaws in Programme for International Student Assessment (PISA) had resulted in an apparent improvement in the attainment of British students, particularly when compared to their Swiss and German counterparts. This paper responds to Prais's criticisms, noting that when Prais's conjectures are tested with empirical data they are not supported. The author argues that many of Prais's criticisms are ...


The Implications Of Halo Effects And Item Dependencies For Objective Measurement, T Mcnamara, Ray Adams Dec 1999

The Implications Of Halo Effects And Item Dependencies For Objective Measurement, T Mcnamara, Ray Adams

Prof Ray Adams

Analysis of data from performance-based language tests presents the problem of dealing appropriately with bundles of items that are linked to particular tasks. The authors use recent developments in Rasch modelling which allow analysis of bundles of items (or sets of ratings) to examine the extent of dependence between items and ratings, and they explore the implications of those dependences on the assessments. Data from two performance assessments are used in two studies of this problem.


Interpreting The Parameters Of A Multidimensional Rasch Model, W Wang, M Wilson, Ray Adams Dec 1999

Interpreting The Parameters Of A Multidimensional Rasch Model, W Wang, M Wilson, Ray Adams

Prof Ray Adams

This chapter introduces the multidimensional random coefficient multinomial logit model (MRCML) which can be applied to multidimensional polytomous test items. Two examples of real data analyses are followed to demonstrates the applications of the MRCML. A detailed discussion is made of interpretation of item parameters, in comparision to those derived from their corresponding unidimensional Rasch measurement models. Finally, it provides some guidelines for constructing scoring matrices and the corresponding models to aid parameter interpretation.


Broadening Assessment, Improving Fairness? Some Examples From School Science, Jan Lokan, Ray Adams, Brian Doig Feb 1999

Broadening Assessment, Improving Fairness? Some Examples From School Science, Jan Lokan, Ray Adams, Brian Doig

Prof Ray Adams

Following a review of equity issues in testing, especially in relation to science, two assessment programmes in school science, each of which was designed both to probe a range of aspects of student performance and for use in a large-scale study, are discussed. Both have been developed and used in the 1990s, one is Australia only and the other in many countries. The programmes and the range of aspects they attempt to assess are described and results are cited for several groups of students. Issues of fairness through broadening of large-scale assessments to include tasks requiring students to construct their ...


International Student Assessment For The New Millennium, Ray Adams Apr 1998

International Student Assessment For The New Millennium, Ray Adams

Prof Ray Adams

The author briefly describes the OECD initiated project, 'Program for International Student Assessment' (PISA), in which over 100,000 15 year old students from 4000 schools in 26 countries will participate. A major focus of the project is the assessment of wider knowledge, skills and competencies in literacy, mathematics and science areas.


Acer Conquest: Generalised Item Response Modelling Software, Margaret Wu, Ray Adams, M Wilson Dec 1997

Acer Conquest: Generalised Item Response Modelling Software, Margaret Wu, Ray Adams, M Wilson

Prof Ray Adams

ACER Conquest is a computer program for fitting item response and latent regression models. It provides a comprehensive and flexible range of item response models to analysts, allowing them to examine the properties of performance assessment, traditional assessments and rating scales. This manual examines ACER ConQuest in detail, includes a tutorial and coverage of technical matters.


Measuring Individual Differences In Change With Multidimensional Rasch Models, W Wang, M Wilson, Ray Adams Dec 1997

Measuring Individual Differences In Change With Multidimensional Rasch Models, W Wang, M Wilson, Ray Adams

Prof Ray Adams

Item response models have been developed to explore change measurement, including those proposed by Fischer and his colleagues. In this article the authors propose another multidimensional Rasch model, the multidimensional random coefficient multinomial logit (MRCML) model. All these models are briefly reviewed and compared. The MRCML can be applied to not only polytomous items but also investigation of variations in item difficulties. Based on variations in difficulties across occasions and items, five kinds of models are proposed. Some simulation studies were conducted to examine parameter recovery of the MRCML model under various testing situations. All the parameters were recovered very ...


The Multidimensional Random Coefficients Multinomial Logit Model, Ray Adams, M Wilson, W Wang Dec 1996

The Multidimensional Random Coefficients Multinomial Logit Model, Ray Adams, M Wilson, W Wang

Prof Ray Adams

A multidimensional Rasch-type item response model, the multidimensional random coefficients multinomial logit model, is presented as an extension to the Adams and Wilson (1996) random coefficients multinomial logit model. The model is developed in a form that permits generalization to the multidimensional case of a wide class of Rasch models, including the simple logistic model, Masters' partial credit model, Wilson's ordered partition model, and Fischer's linear logistic model. Moreover, the model includes several existing multidimensional models as special cases, including Whitely's multicomponent latent trait model, Andersen's multidimensional Rasch model for repeated testing, and Embretson's multidimensional ...


Applications Of Item Response Theory In The Third International Mathematics And Science Study, E Gonzalez, Ray Adams, Marg Wu Dec 1996

Applications Of Item Response Theory In The Third International Mathematics And Science Study, E Gonzalez, Ray Adams, Marg Wu

Prof Ray Adams

The Third International Mathematics and Science Study (TIMSS) is a survey of mathematics and science achievement sponsored by the International Association for the Evaluation of Educational Achievement. With more than 45 participating countries, three target populations, more than thirty languages, 15,000 schools, more than half a million students, TIMSS is the largest and most ambitious international study of educational achievement ever undertaken. Students participating in the assessment were administered one of eight achievement tests that contain items in mathematics and science, using a variety of item formats. As TIMSS is an international study, many audiences will look at the ...


Rasch Models Of Multidimensionality Between Items And Within Items, W Wen-Chung, M Wilson, Ray Adams Dec 1996

Rasch Models Of Multidimensionality Between Items And Within Items, W Wen-Chung, M Wilson, Ray Adams

Prof Ray Adams

This chapter focuses on Rasch measurement and extends it to multidimensional Rasch models. The author first clarifies multidimensionality at two levels: the item level and the test level. With this two way classification, three kinds of tests are formed: undimensional tests, multidimensional between-item tests, and multidimensional within-item tests. The author describes a multidimensional Rasch model, the multidimensional random coefficients multinomial logit (MRCML) model, for multidimensional tests. The model, when applied to multidimensional between-item tests, is more efficient than are unidimensional models in terms of model fit and reliability indices. The model can also be applied to investigate scale interaction of ...


Multilevel Item Response Models: An Approach To Errors In Variables Regression, Ray Adams, M Wilson, Margaret Wu Dec 1996

Multilevel Item Response Models: An Approach To Errors In Variables Regression, Ray Adams, M Wilson, Margaret Wu

Prof Ray Adams

In this article the authors show how certain analytic problems that arise when one attempts to use latent variables as outcomes in regression analyses can be addressed by taking a multilevel perspective on item response modelling.


Bias In Weighted Likelihood Estimation When Using The Rasch Model, D Roberts, Ray Adams Dec 1996

Bias In Weighted Likelihood Estimation When Using The Rasch Model, D Roberts, Ray Adams

Prof Ray Adams

There are three methods of maximum likelihood estimation that are commonly used with Rasch item response models: Joint maximum likelihood, which provides point estimates for both person and item parameters; conditional maximum likelihood, which produces estimates for the item parameters only, and marginal maximum likelihood, which produces estimates of the item parameters and parameters of the distribution from which it has been assumed that the persons have been sampled. A number of approaches have been suggested for obtaining ability estimates for subsequent use in reporting or analysis when either conditional or marginal methods have been used to obtain the item ...


Design Of The Timss Achievement Instruments, Ray Adams, E Gonzalez Dec 1995

Design Of The Timss Achievement Instruments, Ray Adams, E Gonzalez

Prof Ray Adams

No abstract provided.


Quest : The Interactive Test Analysis System, Ray Adams, Siek Toon Khoo Dec 1995

Quest : The Interactive Test Analysis System, Ray Adams, Siek Toon Khoo

Prof Ray Adams

This is a guide to using Quest, a test and questionnaire analysis environment. It includes a control language, and can be used to construct and validate variables based on both dichotomous and polychotomous observations. It scores and analyses such instruments as multiple choice tests, Likert type rating scales, short answer items, and partial credit items.


Student Progress, Charting Of, Geoff Masters, Ray Adams, M Wilson Dec 1993

Student Progress, Charting Of, Geoff Masters, Ray Adams, M Wilson

Prof Ray Adams

This article looks at the charting of students' progress. It examines learning as an active process, conventional achievement testing, testing for conceptual understanding, constructing ordered outcome categories, and collecting observations.


Sex Bias In Asat?, Ray Adams Dec 1983

Sex Bias In Asat?, Ray Adams

Prof Ray Adams

Since 1977 when the Australian Scholastic Aptitude Test was first used in the ACT as a moderating device, there have been differences in the average performance of males and females on the test. This difference in mean group performance has been referred to as a sex bias. This report investigated the nature and the origins of those observed sex differences in ASAT mean scores. Retention rates were investigated to determine the effects of different retention patterns for male and female students on their ASAT scores. Students' attitudes were explored to examine the relationships between sex, attitudes and performance on ASAT ...