Educational Assessment, Evaluation, and Research Commons^™

Literacy And Numeracy Test For Initial Teacher Education Students. Assessment Framework, Australian Council For Educational Research (Acer)

Assessment and Reporting

The Australian Council for Educational Research (ACER) was contracted by the Australian Institute of Teaching and School Leadership (AITSL) to develop the assessment framework and the pools of assessment items for the first two years of implementation of an assessment of aspects of literacy and numeracy for students enrolled in initial teacher education courses. The program is known as the literacy and numeracy test of initial teacher education students (LANTITE). The framework begins by giving some background and context to the development of a literacy and numeracy test for students enrolled in initial teacher education courses. Following this are separate …

Assessment: Getting To The Essence, Geoff N. Masters

Assessment and Reporting

The author argues that assessment in education has become over-conceptualised and overcomplicated, and assessment concepts and terminology introduced over the past half century sometimes now function as impediments to clear thinking and good practice. The one fundamental purpose of assessment in education, he says, is to establish and understand where learners are in an aspect of their learning at the time of assessment. When this is recognised, many supposedly important distinctions become less significant. Currently, however, such distinctions tend to result in fragmentation of the field, with proponents championing one assessment purpose or method while denigrating others. He explains why …

Solomon Islands Standardised Tests Of Achievement (Sista). English And Mathematics. Year 4 And Year 6. Report 1 Main Study 2013, Australian Council For Educational Research (Acer)

Assessment and Reporting

The SISTA program is a key monitoring tool of the Solomon Islands Ministry of Education and Human Resource Development and has the capacity to serve multiple functions to a wide range of educational stakeholders. This report provides analysis of year 4 and year 6 Literacy and Numeracy of SISTA 2013. The SISTA results contain a wealth of data about how well students have responded to items that are indicators of curriculum attainment. The observed improvements in overall performances may be attributed to: a. Better tests and improved alignment of the tests with the target population; and b. In the case …

Towards A Growth Mindset In Assessment, Geoff N. Masters

Assessment and Reporting

The approaches we take to assessing learning, the kinds of tasks we assign and the way we report success or failure at school send powerful messages to students not only about their own learning, but also about the nature of learning itself. Assessment and reporting processes shape student, parent and community beliefs about learning – sometimes in unintended ways.

This essay describes three general approaches to evaluating and providing feedback on the outcomes of learning. Each approach is based on a particular way of thinking about what it means to learn successfully, and each has implications for how students view …

Measure For Measure : A Review Of Outcomes Of School Education In Australia, John Ainley, Eveline Gebhardt

Assessment and Reporting

There have been a plethora of research reports providing information about the achievements of students in Australian schools and how those achievements differ among jurisdictions and among groups of students. However, each report has often been viewed in isolation from other similar studies. This report is intended to provide an integrated appraisal of the results of the international and national achievement surveys conducted since 1994 but with some references to earlier studies. In addition to limiting the task to manageable proportions, this time span corresponds to the widespread introduction of modern measurement methods so that there is a better basis …

Accuracy Of Rasch Model Item Parameter Estimation, Luc T. Le, Ray J. Adams

Assessment and Reporting

This study used Monte Carlo simulations to evaluate the item parameter recovery from ACER ConQuest 3 software (Adams, Wu, & Wilson, 2012) for the dichotomous Rasch model. The authors’ primary focus was the comparison of its estimation methods, joint maximum likelihood (JML), marginal maximum likelihood (MML) with a normal distribution assumption and MML with a discrete distributions assumption when the populations were in fact non-normal. The simulation data sets were generated with two test lengths (10 and 50 items) and four alternative true population distributions for the abilities: normal, bimodal, uniform, and chi-square. As expected, results showed that MML-Normal was …

The Impact Of National And International Assessment Programmes On Education Policy, Particularly Policies Regarding Resource Allocation And Teaching And Learning Practices In Developing Countries, Maura Best, Pat Knight, Petra Lietz, Craig Lockwood, Dita Nugroho, Mollie Tobin

Assessment and Reporting

There is a documented global rise in the number of countries undertaking national learning assessments, as well as international and regional learning assessments. Much of this growth, especially in national learning assessments, has occurred in economically developing countries. Little is known, however, on how these assessments affect education policy and practice in developing countries. This review examined the impact of national and international assessment programmes on education policy, particularly policies regarding resource allocation and teaching and learning practices in developing countries. This particular focus on policies regarding resources and teaching and learning practices stemmed from an observation that, particularly in …

Assessing Student Learning: Why Reform Is Overdue, Geoff N. Masters

Assessment and Reporting

Advances in our understanding of human learning require new approaches to assessing and monitoring student learning. Much assessment thinking has changed little over the past fifty years. The field continues to be dominated by twentieth century introductory textbook concepts, including such dichotomies as formative versus summative assessment, criterion-referenced versus norm-referenced testing, quantitative versus qualitative assessment, informal versus formal assessment – distinctions that often hamper rather than promote clear thinking about assessment. Assessment practice also has changed little over this period. Traditional, high-stakes examinations continue to dominate what is taught and learnt in many of our schools and universities. Greater use …

The Power Of Expectation, Geoff N. Masters

Assessment and Reporting

Success in most fields of endeavour depends on an ability to visualise success. It has long been known that elite athletes mentally rehearse each performance prior to its execution. Advances in neuroscience show why this may be so important: the neurological processes involved in visualising a performance are almost identical to those involved in the performance itself. Indeed, simply watching somebody else perform activates ‘mirror’ neurons in the observer paralleling neuronal activity in the performer. The ability to visualise success and an accompanying belief that success is possible appear to be prerequisites for most forms of human achievement.

Qepr Review Of Progress: Review Of Progress Towards Implementing The Government Response To The Queensland Education Performance Review And Related State School Initiatives In Literacy, Numeracy And Science, Geoff N. Masters

Assessment and Reporting

This review was undertaken at the request of the Director-General of the Queensland Department of Education and Training. It reviews progress and makes recommendations ‘about initiatives aimed at improving the performance of Queensland students in literacy, numeracy and science’. More specifically, the Terms of Reference for the review requested an analysis and recommendations relating to:

• the effectiveness of the initiatives implemented in the Government response to the five recommendations of the Queensland Education Performance Review (QEPR); and
• the effectiveness of identified key initiatives in Queensland state primary schools to improve performance in literacy, numeracy and science.

The review was asked …

Reporting And Comparing School Performances, Geoff N. Masters, Glenn Rowley, John Ainley, Siek Toon Khoo

Assessment and Reporting

This report provides advice on the collection and reporting of information about the performances of Australian schools. The focus is on the collection of nationally comparable data. Two purposes are envisaged: use by education authorities and governments to monitor school performances and, in particular, to identify schools that are performing unusually well or unusually poorly given their circumstances; and use by parents/caregivers and the public to make informed judgements about, and meaningful comparisons of, schools and their offerings. Our advice is based on a review of recent Australian and international research and experience in reporting on the performances of schools. …

A Study Into The Assessment And Reporting Of Employability Skills Of Senior Secondary Students, Gabrielle Matters, David D. Curtis

Assessment and Reporting

In order to investigate the most effective ways of assessing and reporting on the employability skills of senior secondary students, the Commonwealth Department of Education, Science and Training (DEST) commissioned this report. The main activity was to evaluate options for assessing and reporting on eight employability skills against five criteria—validity, reliability, objectivity, feasibility, and usability—and to recommend a preferred approach. This work was undertaken by the Australian Council for Educational Research (ACER) in the period July 2007 to January 2008, during which time the (new) Department of Education, Employment and Workplace Relations (DEEWR) became responsible for the initiative. The starting …

Developing Tests And Questionnaires For A National Assessment Of Educational Achievement, Prue Anderson, George Morgan

Assessment and Reporting

This book addresses the design of two types of data collection instruments: student achievement tests and background questionnaires. Part 1 covers the development of an assessment framework and a test blueprint, item writing, pretesting, and final test layout. Part 2 delineates comparable stages and activities in the construction of background questionnaires, which are used to gather information from students, teachers, head teachers, or parents on variables that might help explain differences in student performance on the achievement test. Part 3 describes how to design a manual for test administration to help ensure that all students take the test under standardised …

Research On The Progressive Achievement Tests And Academic Achievement In Schools, Gerard Fogarty

Assessment and Reporting

This report presents three studies examining research conducted in Queensland on Progressive Achievement Tests (PAT) published by Australian Council for Educational Research (ACER). In study 1 ‘Predictive Validity of the Progressive Achievement Tests for Boys’, a battery of tests was used to assess vocabulary, comprehension, and numeracy skill levels of students commencing study at a regional high school. The tests were used as a screening tool to identify students who might benefit from additional instruction and also to select high achieving students for participation in extension work, maths competitions, and the like. Test results were later correlated with performance in …

Year 12 Curriculum Content And Achievement Standards, Gabrielle Matters, Geoff N. Masters

Assessment and Reporting

This study provides the first Australia-wide picture of what is expected of students taking five subjects—English (including Literature), Mathematics, Chemistry, Physics and Australian History—in the final years of secondary school. The study addresses three broad questions: What is currently taught in these five subjects across Australia? This question is addressed through an analysis of the content of senior school curricula in the five subjects in all Australian states and territories. The study documents similarities and differences in approaches and in the content (subject matter and skills/understandings) emphasised in state and territory curriculum documents. What is the 'essential' content (subject matter …

Australian Certificate Of Education : Exploring A Way Forward, Geoff Masters, Margaret Forster, Gabrielle Matters, Jim Tognolini

Assessment and Reporting

In May 2005 the Department of Education, Science and Training commissioned the Australian Council for Educational Research (ACER) to investigate and report on models and implementation arrangements for an Australian Certificate of Education (ACE). In particular, ACER was asked to investigate four options for the introduction of a new certificate. The current investigation has reviewed existing curriculum, assessment and certification arrangements in the senior years of school in Australia; explored current issues in senior secondary education and examined state and territory responses to these issues; looked at some alternative certification arrangements internationally—including the International Baccalaureate Diploma Program; investigated the use …

The Estimation Of Polytomous Item Response Models With Many Dimensions, Nikolai Volodin, Ray J. Adams

Assessment and Reporting

Identification conditions and an improved estimation method for a D-dimensional mixed coefficients multinomial logit model are discussed. This model is a generalisation of the Adams and Wilson (1997) random coefficients multinomial logit and it can be used to fit multdimensional forms of a wide range of Rasch measurement models. The computational demands of the numerical integration required in fitting such models have limited previous implementations to three and perhaps four-dimensional problems (Glas, 1992; Adams, Wilson and Wang, 1997). This paper illustrates a Monte Carlo integration method that permits the estimation of models with much higher dimensionality. The example in …

Fair And Meaningful Measures? A Review Of Examination Procedures In The Nsw Higher School Certificate, Geoff N. Masters

Assessment and Reporting

In September 1997 the New South Wales government introduced a number of significant reforms to the Higher School Certificate (HSC). The broad objectives of the 'new' HSC, which was implemented for the first time in 2000-01, are to increase the rigour and quality of the HSC curriculum; ensure HSC marks fairly reflect the standards achieved by students; provide more informative reports of students' HSC achievements; and enhance the chances for more equitable educational outcomes. Although the new HSC incorporates reforms of both HSC curricula and methods of assessing and reporting student achievement, the focus of this review is on new …

The Assessments We Need, Geoff N. Masters, Margaret Forster

Assessment and Reporting

Large-scale assessment programs have an important role to play in providing dependable information for educational decision making by policy makers, system managers, school leaders, teachers and parents. But these programs - which include international achievement studies, national surveys and system-wide tests - also convey powerful messages about the kinds of learning valued by educational authorities and can have a profound impact on teaching and learning, particularly if results are reported and compared publicly. For this reason it is essential that large-scale programs are designed to reflect and reinforce learning priorities. This paper argues that large- scale programs are most likely …

Profiles Of Learning. The Basic Skills Testing Program In New South Wales 1989, Geoff N. Masters, Janice Lokan, Brian Doig, Siek Toon Khoo, John Lindsey, Lynette Robinson, Susan Zammit

Assessment and Reporting

The 1989 Basic Skills Testing Program in New South Wales provides the most comprehensive picture yet compiled of literacy and numeracy learning in Australian primary schools. In 1989, some 53,800 Year 6 students in NSW government schools were tested in five aspects of literacy and numeracy. Another 2,300 Year 3 students took part in a pilot study. This book discusses the writing of the tests, the analysis of results, and the reporting of results to parents, teachers and schools. The aim of the basic skills tests is to describe, in positive terms, the skills that students have mastered, to identify …