Education Commons^™

Torch Plus : Tests Of Reading Comprehension Plus, Tom Lumley, Andrew Stephanou

Dr Tom Lumley

A set of reading tests designed to assist teachers of students from Year 3 to Year 10 in their assessment of students' reading comprehension skills.

The Mixed-Coefficients Multinomial Logit Model : A Generalized Form Of The Rasch Model, Ray Adams, Margaret Wu

Prof Ray Adams

Since Rasch's introduction of his item response models, there has been a proliferation of extensions and alternatives, each of which has a different name and different matching software package. The proliferation of models has, in some ways, been a hindrance to practitioners. This paper presents a generalised item response model that provides a unifying framework for a large class of Rasch-type models. The advantages of a single framework include mathematical elegance, generality in a single software package, and a facilitation of the development, testing, and comparison of new models. The unified model is a multidimensional item response model, the specification …

The Effect Of Test-Taker Gender, Audience And Topic On Task Performance In Tape-Mediated Assessment Of Speaking., Tom Lumley, Barry O'Sullivan

Dr Tom Lumley

Performance in tests of spoken language can be influenced by a wide range of features of both task and participants. This article hypothesises that there may be effects on performance attributable to an interaction of variables such as the task topic, the gender of the person presenting the topic and the gender of the candidate. In contrast to previous studies, which have examined speaking tests involving face-to-face interaction, this study considered the issue in a tape-mediated test delivered in a language laboratory, with no interlocutor present, but where stimulus material is presented by one or more speakers, one of whom …

Affective Engagement : A Person-Centred Approach To Understanding The Structure Of Subjective Learning Experiences, Sarah Buckley, Galit Hasen, Mary Ainley

Dr Sarah Buckley

Accounts of students' learning have increasingly emphasised the role of affective engagement in achievement settings. Although most studies have focused on negative emotional experiences such as anxiety, more recent studies have investigated the role of positive emotions. This study examines the structure of students' subjective learning experiences in relation to individual interest profiles. It measured two components of affect: activation as positive arousal that indicates engagement, and valence as an evaluative quality of the students' experience. Senior secondary students (females, N=162) completed measures of individual interests, curiosity and prior knowledge, read three social issues texts and then answered some questions …

Item Format Types And Their Influence On Cross-National Comparisons Of Sudent Performance, Alla Routitsky, Ross Turner

Dr Alla Routitsky

An important decision to be made in the design and implementation of large-scale assessments is the appropriate balance of item types — in particular the balance between objectively coded multiple-choice type items and item types that require coding by trained professionals. Decisions made in this regard depend upon finding an appropriate balance in the light of cost and construct validity concerns. Given the importance of this decision there are surprisingly few international studies that consider the effect of item format on achievement and the results of these studies are not consistent. Some of these studies do, however, illustrate that item …

The Effect Of Test-Taker Sex, Audience And Topic On Task Performance In Tape-Mediated Assessment Of Speaking, Tom Lumley, Barry O'Sullivan

Dr Tom Lumley

There is growing interest in the effect on candidate performance of characteristics of the interlocutor in tests of speaking. A range of variables associated with the interlocutor may cause systematic variation in linguistic performance, and consequently in scores awarded. This paper hypothesises that there may be effects on performance attributable to an interaction of these variables; in the present study, the task topic, the sex of the person presenting the topic and the sex of the candidate. This investigated in the context of a tape-mediated test of speaking, where no interlocutor is actually present; instead, stimulus material is presented by …

Conflicting Perspectives On The Role Of Test Preparation In Relation To Learning, Tom Lumley, Bernadette Stoneman

Dr Tom Lumley

Reports on the development of a learning package for a newly introduced test in Hong Kong, the Graduating Students' Language Proficiency Assessment (GSLPA)-English. Examines reactions of teachers and students to the learning package during piloting of the materials.

Item Panelling, Or Cognitive Walk-Through, Ross Turner

Ross Turner

Enhanced Mathematics Learning: Does Technology Make A Difference?, Katherine Dix

Dr Katherine Dix

Linguistic And Cultural Norms In Language Testing : A Case Study, Annie Brown, Tom Lumley

Dr Tom Lumley

Users of English in Asia may not have the same need to aspire to a ' standard' form of English. It may be more appropriate for them to develop a communicative competence employing the sociolinguistic and cultural norms of the region. Taking such an approach in the development of tests of English proficiency involves rethinking basic assumptions. This paper considers these issues in the context of a test of English proficiency developed for use with English teachers in Indonesia

Specific-Purpose Language Performance Tests : Task And Interaction, Tom Lumley, Annie Brown

Dr Tom Lumley

Oral performance tests designed to assess candidates' ability to function in a particular professional role typically involve role plays which simulate aspects of professional interaction between a candidate, in his/her professional role, and an interlocutor, in the role of client or consumer of the professional services. This paper reports on an investigation into the role which expert professional informants can play in providing post hoc content validation of tasks used in this kind of occupation-specific test, focusing on both the content of the tasks and the behaviour of the interlocutors. The study draws on data from the oral interaction component …

Mapping Abilities And Skill Levels Using Rasch Techniques, Annie Brown, Catherine Elder, Tom Lumley, Tim Mcnamara, Joy Mcqueen

Dr Tom Lumley

The ongoing interest in investigating the use of Item Response Theory (IRT) methods, involving the Rasch model, in language testing research and the development of language tests has encouraged the authors to explore the nature of this approach and to introduce some research on its validation.

Reforming The Assessment Of Student Achievement In The Senior Secondary School, Geoff Masters, P Hill

Prof Geoff Masters AO

The challenge that confronts agencies responsible for assessment and reporting in the senior secondary school is to extend systematic assessment procedures to broader range of learning outcomes than those currently assessed by public examination, to develop methods of reporting which are more descriptive of individual achievement and which provide a better basis for describing and maintaining standards, and to provide results which are sufficiently comparable across schools to enable fair comparisons of applicants for tertiary study. Some recent developments in assessment and reporting practice are considered with a view to identifying methods and approaches capable of satisfying this diverse set …

Banking Non-Dichotomously Scored Items, Geoff Masters, John Evans

Prof Geoff Masters AO

A method for constructing a bank of items scored in two or more ordered response categories is described and illustrated. This method enables multistep problems, rating scale items, question 'clusters', and other items using partial credit scoring to be calibrated and incorporated into an item bank, and it provides a mechanism for computer adaptive testing with items of this type. Procedures are described for calibrating an initial set of items, for testing the fit of items to the underlying measurement model, and for linking new items to an existing item bank. The method is illustrated using items from the Watson-Glaser …

Dicot: Analysing Classroom Tests With The Rasch Model, Geoff Masters

Prof Geoff Masters AO

A computer program (DICOT) for the Rasch analysis of classroom tests is described. Results are presented in a simple, self-explanatory form, and person ability and item difficulty estimates are expressed in a metric like the one with which teachers and parents are already familiar. Person and item fit statistics provide an opportunity to diagnose strengths and weaknesses of individual children and to identify items which are problematic.