Education Commons^™

Do Effect-Size Measures Measure Up?: A Brief Assessment, Anthony J. Onwuegbuzie, Joel R. Levin, Nancy L. Leech

Nancy Leech

Because of criticisms leveled at statistical hypothesis testing, some researchers have argued that measures of effect size should replace the significance-testing practice. We contend that although effect-size measures have logical appeal, they are also associated with a number of limitations that may result in problematic interpretations of them in research on children and adults with learning disabilities (LD). The purpose of the present paper is to provide a framework for reporting and interpreting empirical research findings in LD research. Specifically, we recommend that: (1) researchers apply criteria of both statistical significance and substantive significance to help consumers of research assess …

Determining The Quality Of Assessment Items In Collaborations: Aspects To Discuss To Reach Agreement, Lambert Schuwirth, Jacob Pearce

Dr Jacob Pearce

No abstract provided.

Technological Issues For Computer-Based Assessment, Beno Csapó, John Ainley, Randy Bennett, Thibaud Latour, Nancy Law

Dr John Ainley

This chapter reviews the contribution of new information-communication technologies to the advancement of educational assessment. Improvements can be described in terms of precision in detecting the actual values of the observed variables, efficiency in collecting and processing information, and speed and frequency of feedback given to the participants and stakeholders. The chapter reviews previous research and development in two ways, describing the main tendencies in four continents (Asia, Australia, Europe and the US) as well as summarising research on how technology advances assessment in certain crucial dimensions (assessment of established constructs, extension of assessment domains, assessment of new constructs and …

A Systematic Approach To Literacy Support For First Year Preservice Teachers: Implications For Practice, Pauline Taylor

Associate Professor Pauline Taylor-Guy

Developing Tests And Questionnaires For A National Assessment Of Educational Achievement, Prue Anderson, George Morgan

Prue Anderson

The authors introduce readers to the activities involved in the development of achievement tests, including developing as assessment framework, writing multiple choice and constructed response type items, pretesting, producing test booklets, and handscoring items. A section on questionnaire construction features designing questionnaires, writing questions, coding responses, and linking questionnaire and test score data. The final section covers the development of a test administration manual, selecting test administrators, and contacting sampled schools. A companion CD contains examples of released items from national and international tests, sample questionnaires, and administrative manuals. [Back cover, ed]

Modelling Mathematics Problem Solving Item Responses Using A Multidimensional Irt Model, Margaret Wu, Ray Adams

Prof Ray Adams

This research examined students' responses to mathematics problem- solving tasks and applied a general multidimensional IRT model at the response category level. In doing so, cognitive processes were identified and modelled through item response modelling to extract more information than would be provided using conventional practices in scoring items. More specifically, the study consisted of two parts. The first part involved the development of a mathematics problem-solving framework that was theoretically grounded, drawing upon research in mathematics education and cognitive psychology. The framework was then used as the basis for item development. The second part of the research involved the …

All Is Happening, With Numeracy Included, Dave Tout

David (Dave) Tout

The Adult Literacy and Lifeskills (ALL) Survey (formerly known as the International Life Skills Survey (ILSS)) is a large-scale, comparative survey that goes beyond previous international literacy studies. In addition to the literacy skills measured in the previous International Adult Literacy Survey (IALS), ALL is designed to identify and measure a broader range of skills in the adult population in each participating country. The skills to be directly measured are: prose and document literacy; numeracy; problem solving/analytical reasoning. In addition the assessment will be accompanied by a comprehensive Background Questionnaire, which will collect participant information and indirectly measure two other …

Examining The Evidence : Science Achievement In Australian Schools In Timss 2002, Sue Thomson, Nicole Fleming

Nicole Wernert

Australia, 10030 students in 414 schools participated in the main sample of TIMSS 2002/03. ...Results are reported as average scores with the standard error, as distributions of scores, and as percentages of students who attain the international benchmarks, for countries and specific groups of students within Australia.

Summing It Up : Mathematics Achievement In Australian Schools In Timss 2002, Nicole Fleming, Sue Thomson

Nicole Wernert

This document analyses and interprets the Australian data collected as part of the TIMSS study for Year 4 and Year 8 students.

Anchor Tests, Score Equating And Sex Bias, Geoff Masters

Prof Geoff Masters AO

This paper discusses the use of anchor tests (scaling tests) to bring two or more sets of scores to a common scale. Particular attention is given to the rescaling of school based assessments against an external test or examination and to potential sources of bias in this procedure. The need for routine validity checks is emphasised, and a latent trait approach to constructing a statistical framework for tests and examination score equating is described and illustrated. Bias caused by rescaling school assessments against an inappropriate anchor test is illustrated using a 1984 attempt to rescale students assessments in English against …

Item Discrimination: When More Is Worse, Geoff Masters

Prof Geoff Masters AO

High item discrimination can be a symptom of a special kind of measurement disturbance introduced by an item that gives persons of high ability a special advantage over and above their higher abilities. This type of disturbance, which can be interpreted as a form of item bias, can be encouraged by methods that routinely interpret highly discriminating items as the best items on a test and may be compounded by procedures that weight items by their discrimination. The type of measurement disturbance described and illustrated in this paper occurs when an item is sensitive to individual differences on a second, …

Banking Non-Dichotomously Scored Items, Geoff Masters, John Evans

Prof Geoff Masters AO

A method for constructing a bank of items scored in two or more ordered response categories is described and illustrated. This method enables multistep problems, rating scale items, question 'clusters', and other items using partial credit scoring to be calibrated and incorporated into an item bank, and it provides a mechanism for computer adaptive testing with items of this type. Procedures are described for calibrating an initial set of items, for testing the fit of items to the underlying measurement model, and for linking new items to an existing item bank. The method is illustrated using items from the Watson-Glaser …