Education Commons^™

Pat Science : Getting Good Assessment Down Pat, Geoff Masters

Prof Geoff Masters AO

A new school assessment resource in science from the Australian Council for Educational Research (ACER) has just been launched, following more than two years of development and trial testing. Called PAT Science, the new addition to the Progressive Achievement Tests series complements PAT Reading and PAT Maths, which are now widely used in Australian schools. PAT Science fills an identified need for better classroom information about the progress students make in their science learning between Year 3 and Year 10. By the time they reach 15 years of age, 13 percent of Australian students are classified as 'at risk' of …

South Africa And Uae Student Performance Progression In Ibt Mathematics And Science, Van Nguyen

Dr Van Nguyen

No abstract provided.

Using Eighth Grade Georgia Criterion-Referenced Competency Tests To Predict Student Achievement On The Georgia End Of Course Tests, Janice Darnell

Doctoral Dissertations and Projects

The purpose of this correlational study was to examine Georgia's Criterion-Referenced Competency Test (CRCT) scores of 8th grade students and End of Course Test (EOCT) scores of the same students as 9th graders in the areas of language arts and mathematics to test the theory that a relationship exists between the two tests. The study also examined the 8th grade CRCT scores as being predictors of the 9th grade EOCT scores. Three cycles, or classes, of 8th grade CRCT scores and corresponding EOCT scores were used. The study used data from a small semi-rural school district in northeast Georgia. The …

Is There A Relationship Between Chemistry Performance And Question Type, Question Content And Gender?, Ross Hudson

Dr Ross Hudson

This research inquires into the effectiveness of the two predominant forms of questions - multiple-choice questions and short-answer questions - used in the State University Entrance Examination for Chemistry including the relationship between performance and gender. It examines not only the style of question but also the content type examined (recall and application questions). The research involves class trial testing of students with structured questions that examine the same material content with each type of question (multiple-choice or short-answer) and also examines the different type of content (recall or application) and finally the influence of student gender. Rasch analysis of …

Cartoons And Linguistic Context, Brad Jackel

Dr Brad Jackel

This paper presents the results of a trial conducted during the development of The Graduate Australian Medical School Admission Test (GAMSAT) 2006, where two versions of a cartoon unit were trialled; in both cases the cartoon and the question itself were identical. The only difference was in the way the cartoon was introduced within the test. Psychometrically, across the 650 candidates involved in the trial, one version ‘worked’ very well, the other not at all.

Listening For The Squeaky Wheel: Designing Distance Writing Program Assessment, Virginia M. Tucker

English Faculty Publications

Distance writing programs still struggle with assessment strategies that can evaluate student writing as well as their ability to communicate about that writing with peers at a distance. This article uses Kim, Smith and Maeng's 2008 distance education program assessment scheme to evaluate a single distance writing program at Old Dominion University. The program's specific assessment needs include the ability to determine how well students are developing expert insider prose and working together as a virtual community. Kim, Smith and Maeng's assessment scheme was applied to six courses within the writing program, revealing that programmatic assessment weaknesses included providing varied …

Technological Issues For Computer-Based Assessment, Beno Csapó, John Ainley, Randy Bennett, Thibaud Latour, Nancy Law

Dr John Ainley

This chapter reviews the contribution of new information-communication technologies to the advancement of educational assessment. Improvements can be described in terms of precision in detecting the actual values of the observed variables, efficiency in collecting and processing information, and speed and frequency of feedback given to the participants and stakeholders. The chapter reviews previous research and development in two ways, describing the main tendencies in four continents (Asia, Australia, Europe and the US) as well as summarising research on how technology advances assessment in certain crucial dimensions (assessment of established constructs, extension of assessment domains, assessment of new constructs and …

Which Form Of Assessment In A Chemistry Examination Best Describes Student Understanding?, Ross Hudson

Dr Ross Hudson

No abstract provided.

Using Individual And Group Multiple-Choice Quizzes To Deepen Students' Learning, Sophie M. Sparrow

Law Faculty Scholarship

For years, I was highly skeptical about using multiple-choice questions to assess law students' learning.' Clients, after all, do not ask lawyers to solve multiple-choice problems. I have realized, however, that multiple-choice quizzes can be a highly effective technique to include in any doctrinal class. Well-designed multiple-choice quizzes can help students in any size class learn foundational doctrine, provide feedback to teachers and students, develop students' interpersonal skills, and prepare students for the bar exam. Having used multiple-choice quizzes in first year and upper-level courses for several years, I now value multiple-choice quizzes as an effective first step in preparing …

Multiple-Choice Questions Compared To Short-Answer Response : Which Assesses Understanding Of Chemistry More Effectively?, Ross Hudson

Dr Ross Hudson

The research inquires into the effectiveness of the two predominant forms of questions that are used on the Victorian Certificate of Education (VCE) Victoria Chemistry examination. These are multiple- choice questions and short-answer questions. This research examines not only the style of chemistry question but also the content type examined (recall and application questions) along with gender differences in students' responses to such questions. The research involved three phases, i) analysis of five years results from the VCE Chemistry examinations, ii) class trial testing students of both genders with structured questions that examined the same material content with each type …

Pisa : Frequently Answered Criticisms, Ray Adams

Prof Ray Adams

Studies such as PISA that attempt to compare outcomes across educational systems are expensive and difficult to implement. Further, the results of such studies are routinely criticized by educational commentators - particularly when the results are not consistent with their preconceived ideas about the relative merits and efficiencies of various educational practices and systems. This chapter discusses what is done to ameliorate the threats to the validity of PISA in five areas often targeted by commentators and reviewers as sources of invalidity in international comparisons. The five areas discussed are: (1) sampling - are the samples of students who undertake …

What Information From Pisa Is Useful For Teachers? How Can Pisa Help Our Students To Become More Proficient?, Juliette Mendelovits, Dara Searle, Tom Lumley

Dr Tom Lumley

A frequent objection to large-scale testing programs, both national and international, is that they are used as an instrument of control, rather than as a means of providing information to effect change. Moreover, concerns about large-scale testing often take the form of objection to the specific characteristics of the assessments as being prescriptive and proscriptive, leading to a narrowing of the curriculum and the spectre of 'teaching to the test' to the exclusion of more important educational content. Taking PISA reading literacy as its focus, this paper proposes, on the contrary, that a coherent assessment system is valuable in so …

Patscience : Progressive Achievement Tests In Science, Ross Hudson, Ron Martin, Daniel Urbach, Stavroula Zoumboulis

Dr Ross Hudson

The ACER progressive achievement tests in science are for use in Australian schools to provide information to teachers about the level of achievement attained by their students in the concepts, skills and processes of science.

Learning Mathematics For Life : A Perspective From Pisa, Jan De Lange, Alla Routitsky, Kaye Stacey, Ross Turner, Margaret Wu, Andreas Schleicher, Claire Shewbridge, Pablo Zoido, Nicola Clements

Dr Alla Routitsky

People from many countries have expressed interest in the tests students take for the Programme for International Student Assessment (PISA). This book examines the link between the PISA test requirements and student performance. It focuses specifically on the proportions of students who answer questions correctly across a range of difficulty. The questions are classified by content, competencies, context and format, and the connections between these and student performance are then analysed. This analysis has been carried out in an effort to link PISA results to curricular programs and structures in participating countries and economies. Results from the student assessment reflect …

All Is Happening With Numeracy : A New International Survey Of Adults' Numeracy And Mathematics Skill Levels., Dave Tout

David (Dave) Tout

An international survey of the numeracy abilities of adults was part of the Adult Literacy and Lifeskills (ALLS) survey conducted in Australia in 2006 and the start of 2007. This paper includes information on the construct behind the numeracy component of the survey, including the complexity scheme that attempts to predict the difficulty of test items, and reports on the initial data from the first wave of countries, highlighting the important role that numeracy plays in both human and social capital terms. First Australian results are due out in late 2007.

Torch Plus : Tests Of Reading Comprehension Plus, Tom Lumley, Andrew Stephanou

Dr Tom Lumley

A set of reading tests designed to assist teachers of students from Year 3 to Year 10 in their assessment of students' reading comprehension skills.

The Mixed-Coefficients Multinomial Logit Model : A Generalized Form Of The Rasch Model, Ray Adams, Margaret Wu

Prof Ray Adams

Since Rasch's introduction of his item response models, there has been a proliferation of extensions and alternatives, each of which has a different name and different matching software package. The proliferation of models has, in some ways, been a hindrance to practitioners. This paper presents a generalised item response model that provides a unifying framework for a large class of Rasch-type models. The advantages of a single framework include mathematical elegance, generality in a single software package, and a facilitation of the development, testing, and comparison of new models. The unified model is a multidimensional item response model, the specification …

The Effect Of Test-Taker Gender, Audience And Topic On Task Performance In Tape-Mediated Assessment Of Speaking., Tom Lumley, Barry O'Sullivan

Dr Tom Lumley

Performance in tests of spoken language can be influenced by a wide range of features of both task and participants. This article hypothesises that there may be effects on performance attributable to an interaction of variables such as the task topic, the gender of the person presenting the topic and the gender of the candidate. In contrast to previous studies, which have examined speaking tests involving face-to-face interaction, this study considered the issue in a tape-mediated test delivered in a language laboratory, with no interlocutor present, but where stimulus material is presented by one or more speakers, one of whom …

Affective Engagement : A Person-Centred Approach To Understanding The Structure Of Subjective Learning Experiences, Sarah Buckley, Galit Hasen, Mary Ainley

Dr Sarah Buckley

Accounts of students' learning have increasingly emphasised the role of affective engagement in achievement settings. Although most studies have focused on negative emotional experiences such as anxiety, more recent studies have investigated the role of positive emotions. This study examines the structure of students' subjective learning experiences in relation to individual interest profiles. It measured two components of affect: activation as positive arousal that indicates engagement, and valence as an evaluative quality of the students' experience. Senior secondary students (females, N=162) completed measures of individual interests, curiosity and prior knowledge, read three social issues texts and then answered some questions …

Aligning Paper Tests With Multimedia Instruction, Scott L. Howell

Faculty Publications

Although the "click-and-point" virtual classrooms of today hardly resemble the brick-and-mortar classrooms of yesterday, one thing seems to not have changed: the prevalence of paper-based tests. Paper-based tests have been the staple of education for centuries and will most likely persist for many years to come. This article explores some of the issues surrounding the growing chasm between the way students are now taught and how they are still tested from three perspectives: researcher, student, and teacher.

Item Format Types And Their Influence On Cross-National Comparisons Of Sudent Performance, Alla Routitsky, Ross Turner

Dr Alla Routitsky

An important decision to be made in the design and implementation of large-scale assessments is the appropriate balance of item types — in particular the balance between objectively coded multiple-choice type items and item types that require coding by trained professionals. Decisions made in this regard depend upon finding an appropriate balance in the light of cost and construct validity concerns. Given the importance of this decision there are surprisingly few international studies that consider the effect of item format on achievement and the results of these studies are not consistent. Some of these studies do, however, illustrate that item …

Malawi Teachers' Knowledge Of And Attitudes Towards Standardized Tests, Leah Kaira

Master's Capstone Projects

Standardized tests play a great role in education. In Malawi, results of these tests are used to make: critical decisions about individuals, hence the need for students to be well prepared. Students' performance can be influenced by their teachers' attitude toward and knowledge of standardized tests. This survey was designed to establish Malawian teachers· knowledge of standardized tests and their attitude towards these tests. While the teachers displayed positive attitudes, it was found that they would benefit from more knowledge about standardized tests.

The Effect Of Test-Taker Sex, Audience And Topic On Task Performance In Tape-Mediated Assessment Of Speaking, Tom Lumley, Barry O'Sullivan

Dr Tom Lumley

There is growing interest in the effect on candidate performance of characteristics of the interlocutor in tests of speaking. A range of variables associated with the interlocutor may cause systematic variation in linguistic performance, and consequently in scores awarded. This paper hypothesises that there may be effects on performance attributable to an interaction of these variables; in the present study, the task topic, the sex of the person presenting the topic and the sex of the candidate. This investigated in the context of a tape-mediated test of speaking, where no interlocutor is actually present; instead, stimulus material is presented by …

Conflicting Perspectives On The Role Of Test Preparation In Relation To Learning, Tom Lumley, Bernadette Stoneman

Dr Tom Lumley

Reports on the development of a learning package for a newly introduced test in Hong Kong, the Graduating Students' Language Proficiency Assessment (GSLPA)-English. Examines reactions of teachers and students to the learning package during piloting of the materials.

Item Panelling, Or Cognitive Walk-Through, Ross Turner

Ross Turner

Enhanced Mathematics Learning: Does Technology Make A Difference?, Katherine Dix

Dr Katherine Dix

Linguistic And Cultural Norms In Language Testing : A Case Study, Annie Brown, Tom Lumley

Dr Tom Lumley

Users of English in Asia may not have the same need to aspire to a ' standard' form of English. It may be more appropriate for them to develop a communicative competence employing the sociolinguistic and cultural norms of the region. Taking such an approach in the development of tests of English proficiency involves rethinking basic assumptions. This paper considers these issues in the context of a test of English proficiency developed for use with English teachers in Indonesia

Acer Quest: The Interactive Test Analysis System. Version 2.1., Raymond J. Adams, Siek-Toon Khoo

Measurement and statistics

This is a guide to using Quest. Quest offers a comprehensive test and questionnaire analysis environment by providing a data analyst with access to the most recent developments in Rasch measurement theory, as well as a range of traditional analysis procedures. It includes an easy to use control language with flexible and informative output. Quest can be used to construct and validate variables based on both dichotomous and polychotomous observations. It scores and analyses such instruments as multiple choice tests, Likert type rating scales, short answer items, and partial credit items.

Download the legacy ACER Quest software here

Download the …

Specific-Purpose Language Performance Tests : Task And Interaction, Tom Lumley, Annie Brown

Dr Tom Lumley

Oral performance tests designed to assess candidates' ability to function in a particular professional role typically involve role plays which simulate aspects of professional interaction between a candidate, in his/her professional role, and an interlocutor, in the role of client or consumer of the professional services. This paper reports on an investigation into the role which expert professional informants can play in providing post hoc content validation of tasks used in this kind of occupation-specific test, focusing on both the content of the tasks and the behaviour of the interlocutors. The study draws on data from the oral interaction component …

Mapping Abilities And Skill Levels Using Rasch Techniques, Annie Brown, Catherine Elder, Tom Lumley, Tim Mcnamara, Joy Mcqueen

Dr Tom Lumley

The ongoing interest in investigating the use of Item Response Theory (IRT) methods, involving the Rasch model, in language testing research and the development of language tests has encouraged the authors to explore the nature of this approach and to introduce some research on its validation.