Education Commons^™

Reduce Randomly Guessing Effects In A University Generic Skills Test, Luc T. Le

Dr Luc Tu Le

Effect Of Trial Items On Candidate Performances In A Large-Scale Postgraduate Medical Selection Test, Luc T. Le

Dr Luc Tu Le

Evaluate Item Dependency In An Aptitude Reasoning Test, Luc T. Le

Dr Luc Tu Le

Scaling Procedures For Icils Test Items, Eveline Gebhardt, Wolfram Schulz

Dr Wolfram Schulz

This chapter describes the procedures used to analyze and scale the ICILS test items that were administered to measure students’ computer and information literacy (CIL).

Assessing Creative Thinking Skills - An Interdisciplinary Approach, Mark Butler

Mark Butler

The Australian Council for Educational Research (A.C.E.R.) is a not-for-profit organisation that works with Australian and international governments to develop and improve learning and assessment practices. In 2012 A.C.E.R. was contracted by the Victorian government to undertake research in the assessment of critical and creative thinking skills for students aged 8 to 18. Over 300 test items were developed for this purpose and these items were used to form a developmental scale for thinking. The research highlights the how critical and creative thinking skills can be incorporated into lessons which transcend traditional subject boundaries. This presentation focuses on the process …

Scaling Procedures For Iccs Test Items, Wolfram Schulz, Julian Fraillon

Julian Fraillon

This chapter describes the procedures used to analyse and scale the International Civic and Citizenship Education Study (ICCS) international and regional test items that were administered to measure students’ civic knowledge. The chapter covers the following topics: the scaling model used to analyze and scale the test items; test coverage and item dimensionality; assessment of item fit; assessment of scorer reliabilities for open-ended items; differential item functioning by gender; review of cross-national measurement equivalence; international item adjudication; international item calibration and test reliability; international ability estimates (plausible values and weighted likelihood estimates); estimation of changes in civic content knowledge between …

Scaling Procedures For Iccs Test Items, Wolfram Schulz, Julian Fraillon

Dr Wolfram Schulz

This chapter describes the procedures used to analyse and scale the International Civic and Citizenship Education Study (ICCS) international and regional test items that were administered to measure students’ civic knowledge. The chapter covers the following topics: the scaling model used to analyze and scale the test items; test coverage and item dimensionality; assessment of item fit; assessment of scorer reliabilities for open-ended items; differential item functioning by gender; review of cross-national measurement equivalence; international item adjudication; international item calibration and test reliability; international ability estimates (plausible values and weighted likelihood estimates); estimation of changes in civic content knowledge between …

A Three-Way Classification Of Sources Of Item Difficulty In Tests And Examinations, Gabrielle Matters

Dr Gabrielle Matters

What do test takers mean when they say ‘this item is difficult’? What do test analysts mean when they say ‘this item is difficult’? The answer to the first question comes out of experience. The answer to the second question comes out of empirics. The notion of difficulty covers a considerable diversity of sources, materials and methods. Test analysts seem obliged to collapse all senses of difficulty under one heading and so it might be useful to attempt a classification or typo logy of some of the possible sources of difficulty in test items. This presentation describes such a system …

Pisa Mathematics - An Insider's View, Ross Turner

Ross Turner

Pisa Science 2006 : International Results, John Cresswell

Dr John Cresswell

Throughout the world education authorities want to know the capacities that their students develop during their formative years at school. They want to know to what extent students have learned fundamental scientific concepts and theories and how well they can solve real-life problems involving science and technology. In order to provide answers to these questions, PISA provides an international benchmark. Unlike many traditional assessments of student performance in science, PISA is not limited to measuring students' mastery of specific science content. Instead, it measures the capacity of students to identify scientific issues, explain phenomena scientifically and use scientific evidence. These …

Developing Mathematical Literacy, Ross Turner, L Steen, H Burkhardt

Ross Turner

Modelling And Applications In Pisa, Ross Turner

Ross Turner

Response To 'Cautions On Oecd's Recent Educational Survey (Pisa), Ray Adams

Prof Ray Adams

This paper refers to the work of S J Prais who questioned the outcomes of the Organisation for Economic Cooperation and Development's PISA survey of the reading, mathematics and science attainments of fifteen year-olds. Prais suggested that methodological flaws in Programme for International Student Assessment (PISA) had resulted in an apparent improvement in the attainment of British students, particularly when compared to their Swiss and German counterparts. This paper responds to Prais's criticisms, noting that when Prais's conjectures are tested with empirical data they are not supported. The author argues that many of Prais's criticisms are due to an incomplete …

Better Than Beige : Designing Assessment Tasks To Enhance Learning And Measure Growth In The Early Years Of School, Prue Anderson, Marion Meiers

Prue Anderson

'I think of our school as beige. We tend to be the average of 'like' schools. Can we do better than that?' In order to promote growth in student learning and demonstrate achievement schools require rich assessment tasks that enhance the teaching and learning cycle and provide a valid and reliable measure of growth. This paper examines the five linked sets of assessment tasks that have been developed to progressively measure growth in literacy and numeracy in the first three years of school. The tasks were designed for the ACER Longitudinal Literacy and Numeracy Survey (LLANS). The tasks are based …

The Implications Of Halo Effects And Item Dependencies For Objective Measurement, T Mcnamara, Ray Adams

Prof Ray Adams

Analysis of data from performance-based language tests presents the problem of dealing appropriately with bundles of items that are linked to particular tasks. The authors use recent developments in Rasch modelling which allow analysis of bundles of items (or sets of ratings) to examine the extent of dependence between items and ratings, and they explore the implications of those dependences on the assessments. Data from two performance assessments are used in two studies of this problem.

Interpreting The Parameters Of A Multidimensional Rasch Model, W Wang, M Wilson, Ray Adams

Prof Ray Adams

This chapter introduces the multidimensional random coefficient multinomial logit model (MRCML) which can be applied to multidimensional polytomous test items. Two examples of real data analyses are followed to demonstrates the applications of the MRCML. A detailed discussion is made of interpretation of item parameters, in comparision to those derived from their corresponding unidimensional Rasch measurement models. Finally, it provides some guidelines for constructing scoring matrices and the corresponding models to aid parameter interpretation.

Item Panelling, Or Cognitive Walk-Through, Ross Turner

Ross Turner

Acer Conquest: Generalised Item Response Modelling Software, Margaret Wu, Ray Adams, M Wilson

Prof Ray Adams

ACER Conquest is a computer program for fitting item response and latent regression models. It provides a comprehensive and flexible range of item response models to analysts, allowing them to examine the properties of performance assessment, traditional assessments and rating scales. This manual examines ACER ConQuest in detail, includes a tutorial and coverage of technical matters.

Design Of The Timss Achievement Instruments, Ray Adams, E Gonzalez

Prof Ray Adams

No abstract provided.

The Analysis Of Partial Credit Scoring, Geoff Masters

Prof Geoff Masters AO

This article discusses a range of issues in the practical application of an item response theory (IRT) method for partial credit scoring. After a brief discussion of partial credit scoring as an alternative to right-wrong scoring in the measurment of educational achievement, an IRT model for partial credit analysis is developed and described. This model is presented as a straightforward and logical application of Rasch's dichotomous model to a sequence of ordered response alternatives. The distinctive nature of the item parameters in the model is described and these parameters are contrasted with two more familiar sets of parameters: Thurstone thresholds …

Item Discrimination: When More Is Worse, Geoff Masters

Prof Geoff Masters AO

High item discrimination can be a symptom of a special kind of measurement disturbance introduced by an item that gives persons of high ability a special advantage over and above their higher abilities. This type of disturbance, which can be interpreted as a form of item bias, can be encouraged by methods that routinely interpret highly discriminating items as the best items on a test and may be compounded by procedures that weight items by their discrimination. The type of measurement disturbance described and illustrated in this paper occurs when an item is sensitive to individual differences on a second, …

Constructing An Item Bank Using Partial Credit Scoring, Geoff Masters

Prof Geoff Masters AO

A method for banking test items scored in several ordered response categories is described. Each item is seen as an ordered sequence of steps, and test forms are equated using the estimated difficulties of the steps in their shared items. Procedures for analyzing the internal consistency of individual links and for analyzing the coherence of an entire linking structure are described. The methodology is used to link six forms of a mathematics problem solving test.

Defining A 'Fear-Of-Crime' Variable: A Comparison Of Two Rasch Models, Geoff Masters

Prof Geoff Masters AO

No abstract provided.