Educational Assessment, Evaluation, and Research Commons^™

Reforming Educational Assessment: Imperatives, Principles And Challenges, Geoff N. Masters

Australian Education Review

AER 57 reviews research into assessment, especially in schools; it analyses the pivotal role of assessment in learning and argues for its reconceptualisation by practitioners and policy makers to better support learning. The genesis of this AER was the ACER Research Conference Assessment and Student Learning: Collecting, interpreting and using data to inform teaching, held in Perth in August 2009.

Section 1 outlines some current pressures for assessment reform, introduces the concept of a learning assessment system designed to establish where learners are in their progress within an empirically mapped domain of learning, and sketches a set of design …

Formative Evaluation Of Textbooks And Workbooks In South Africa, Rachel Outhred, Adrian Beavis, Catherine Stubberfield, Jenny Wilkinson, Martin Murphy, David Kelly

Program monitoring and evaluation

Textbooks and workbooks have long been considered a critical challenge within the South African education system. The Department of Basic Education, South Africa (DBE) undertook a major initiative to provide Mathematics and Language workbooks to learners in order to accelerate progress towards Education for All, in terms of access to and quality of education. The DBE, with support from UNICEF, commissioned the Australian Council for Educational Research (ACER) to undertake an independent formative evaluation of workbooks and textbooks in South Africa in the period April 2012 – February 2013. The evaluation of the Workbook project was formative. The study was …

Professional Certification: Promoting And Recognising Successful Teaching Practices, Lawrence C. Ingvarson

Teaching standards and teacher evaluation

This paper focuses on the challenge of developing a system forrecognising and rewarding accomplished teachers operated by the teaching profession – a standards-based professional learning and certification system. While the focus is on recent Australian experience, it draws on the experience of several countries that have sought to reform teacher career structures and pay systems so that there is a closer alignment between career progression and increasing expertise. Over the past twenty years or so, teacher associations in several countries have demonstrated that the profession has the ability to reach a consensus on teaching standards without imposing uniformity of teaching …

Iccs 2009 Encyclopedia : Approaches To Civic And Citizenship Education Around The World, John Ainley, Wolfram Schulz, Tim Friedman

Civics and Citizenship Assessment

The International Civic and Citizenship Education Study (ICCS 2009) investigated the ways in which countries prepare their young people to undertake their roles as citizens. It studied student knowledge and understanding of civics and citizenship as well as student attitudes, perceptions, and activities related to civics and citizenship. ICCS also examined differences among countries in relation to these outcomes of civic and citizenship education, and it explored how differences across countries relate to student characteristics, school and community contexts, and national characteristics. Thirty-eight countries participated in ICCS 2009. Among these were five from Asia, 26 from Europe, six from Latin …

Creating An Effective School For Aboriginal And Torres Strait Islander Students, Gina Milgate, Brian Giles-Browne

Indigenous Education Research

In Australia, the Collegial Snapshot (CSS) process has been a culturally safe, engaging and empowering way for Aboriginal and Torres Strait Islander students and their parents and carers to share their insights and ideas about what makes an effective school. The Aboriginal and Torres Strait Islander parents and carers through the Collegial Snapshot Process identified six key variables that make an effective school for their child including the Cultural Environment, Quality of Teachers, Community Engagement, Student Health and Wellbeing, Curriculum and School Leadership. The wealth of data collected from Aboriginal and Torres Strait Islander students and parents and carers complemented …

Making A Difference : Improving Outcomes For Indigenous Learners, Michele Lonsdale

Indigenous Education Research

This publication highlights ACER’s contributions to Indigenous Educational research over the past decade. This contribution has included research, policy analysis, program evaluation, professional development, and the development of assessment tools and other resources.

National Report On Social Equity In Vet 2013, Sheldon Rothman, Chandra Shah, Catherine Underwood, Julie Mcmillan, Justin Brown, Phillip Mckenzie

Vocational, adult and professional education research

This report is the first National Report on Social Equity in VET. It has been developed by the National VET Equity Advisory Council (NVEAC) to provide baseline information on the participation, achievement and transitions from the Australian Vocational Education and Training (VET) system for six groups in the Australian population: Indigenous Australians; people with a disability; people from a culturally and linguistically diverse background; people living in remote areas; people from low socioeconomic status backgrounds; and women. The report also provides information on the experience in VET of a further five groups who may be experiencing difficult life chances …

Reconciliation Action Plan, Australian Council For Educational Research (Acer)

Indigenous Education Research

ACER’s Reconciliation Action Plan (RAP) was prepared by a Working Group including the Chair of the Advisory Committee on Indigenous Education (ACIE). This document has been endorsed by Reconciliation Australia. The RAP demonstrates ACER’s commitment to the reconciliation process and identifies how ACER intends to contribute to the RAP priorities over the next few years. The focus is on how ACER intends to work to create a culture of respect, to build more effective relationships and to create improved employment opportunities for Aboriginal and Torres Strait Islander people.

Queensland Teaching And Learning Audits 2010–2012, Gabrielle Matters, Yvana Jones

School and system improvement

This paper describes the teaching and learning audits introduced in Queensland government schools in the period 2010-2012. The teaching and learning audits were based on an audit tool developed by the Australian Council for Educational Research (ACER) for the Queensland Department of Education and Training (DET). The audit tool was a precursor to the National School Improvement Tool and, in 2010-2012, consisted of the first eight domains of the final tool. The Queensland experience provides a valuable case study of one education system’s use of the tool as a basis for external school reviews, school self-evaluations and school improvement planning.

Teach For Australia Pathway: Evaluation Report Phase 3 Of 3, Paul R. Weldon, Phillip Mckenzie, Elizabeth Kleinhenz, Kate Reid

Initial Teacher education

This report is the final of three reports of the evaluation of the Teach for Australia (TFA) Pathway, a pilot of an alternative approach to teacher education in Australia. The evaluation was undertaken by the Australian Council for Educational Research (ACER) over the period 2010 to 2012. The basic design of the TFA Pathway is as follows: 1. High-achieving university graduates are recruited nationally. Applicants are subject to a rigorous recruitment process and are selected on the basis of qualities and skills suitable to the teaching profession, and the possession of a genuine desire to reduce educational disadvantage. 2. Selected …

Footprints In Time : Who Am I? And Renfrew Word Finding Vocabulary Test : Report On Wave 2 Data, Sarah Buckley, Catherine Underwood, Nola Purdie

Indigenous Education Research

This report presents the results of administration of the Who Am I? and the Renfrew Word Finding Vocabulary Test for the LSIC Wave 2 data collection in 2009. Who am I? is a developmental assessment that requires the child to write their name, copy shapes, write letters, numbers and words in a small booklet, with simple instructions and encouragement from the interviewer. Who am I? is not language dependent and is suitable for children with limited English. The assessment takes about 10 minutes to complete and is suitable for preschool children and children in the first two years of …

Startsmart Impact Report, Commonwealth Bank Foundation

Program monitoring and evaluation

The Commonwealth Bank Foundation’s school-based, financial education program, StartSmart, was developed with the assistance of education experts and launched in 2007. The StartSmart program – is a series of interactive, financial literacy workshops delivered in classrooms by a team of facilitators which works in partnership with school teachers to deliver effective financial education. The program extends to students in primary, secondary and vocational education settings. The Commonwealth Bank Foundation engaged the Australian Council for Educational Research (ACER) to undertake an evaluation of the StartSmart program. The purpose of this research was to measure the impact of StartSmart on students’ financial …

Partnering For School Improvement : Case Studies Of School-Community Partnerships In Australia, Sharon Clerke

Program monitoring and evaluation

The case studies in this booklet illustrate the creative ways in which Australian schools are responding to local needs by establishing and building partnerships with community organisations and businesses. These partnerships have been formed to share resources, both human and material. They are providing students with access to breakfast clubs, homework and tutoring programs, after-school fitness and sporting activities, and other programs designed to re-engage students. Local businesses that want students to know more about employer expectations and the realities of the workplace are providing work placements and work experience, traineeships and apprenticeships. Some schools are partnering with other educational …

International Computer And Information Literacy Study: Assessment Framework, Julian Fraillon, Wolfram Schulz, John Ainley

ICT - Digital Literacy

The purpose of the International Computer and Information Literacy Study 2013 (ICILS 2013) is to investigate, in a range of countries, the ways in which young people are developing computer and information literacy (CIL) to support their capacity to participate in the digital age. To achieve this aim, the study will assess student achievement through an authentic computer-based assessment of CIL administered to students in their eighth year of schooling. It will also collect and report on analyses of data about student use of computers and other digital devices as well as students’ attitudes toward the use of computers and …

The Impact Of National And International Assessment Programmes On Education Policy, Particularly Policies Regarding Resource Allocation And Teaching And Learning Practices In Developing Countries, Maura Best, Pat Knight, Petra Lietz, Craig Lockwood, Dita Nugroho, Mollie Tobin

Assessment and Reporting

There is a documented global rise in the number of countries undertaking national learning assessments, as well as international and regional learning assessments. Much of this growth, especially in national learning assessments, has occurred in economically developing countries. Little is known, however, on how these assessments affect education policy and practice in developing countries. This review examined the impact of national and international assessment programmes on education policy, particularly policies regarding resource allocation and teaching and learning practices in developing countries. This particular focus on policies regarding resources and teaching and learning practices stemmed from an observation that, particularly in …

2012 University Experience Survey National Report, Ali Radloff, Hamish Coates, Rebecca Taylor, Richard James, Kerri-Lee Krause

Higher education research

How students experience university plays a major role in their academic, personal and professional success. Over the last decade Australian universities and governments have placed considerable emphasis on key facets of the student experience ecology such as skills development, student engagement, quality teaching, student support, and learning resources. This report discusses a 2012 project that furnished a new national architecture for collecting feedback on understanding the improving the student experience. The University Experience Survey (UES) has been developed by the Australian Government to provide a new national platform for measuring the quality of teaching and learning in Australian higher education. …

Aqtf Quality Indicator Service : 2011-12 Report, Acer

Vocational, adult and professional education research

In 2007 ACER lead a sector-side development, deployment and support of a set of instruments and reporting tools to underpin a new data-driven and outcomes-focused approach to quality assurance in Australia’s vocational education and training sector. The AQTF 2007 Quality Indicator survey resources provide building blocks for a wide range of conversations about the quality of vocational education and training. Such feedback is used in education systems and organisations to obtain information on the quality of education and training.

Broadly speaking, collecting information about performance provides a valuable source of data for RTOs to:

◗◗ gauge how well they are …

Evaluation Of The Assessment And Rating Process Under The National Quality Standard For Early Childhood Education And Care And School Age Care, Sheldon Rothman, David Kelly, Bridie Raban, Mollie Tobin, Jocelyn Cook, Kate O’Malley, Clare Ozolins, Meredith Bramich

Early Childhood Education

This evaluation of the assessment and rating process for early childhood education and care and school age care services had as its focus the validity and reliability of the process. In particular, do the items reviewed with the Assessment and Rating Instrument provide consistent and replicable measures? Would the judgements made by one authorised officer be made by other authorised officers reviewing the same service? Does the process—including use of the Instrument—allow distinctions between rating levels?

The evaluation was undertaken by the Australian Council for Educational Research (ACER), which analysed assessment and rating data from both draft and final reports; …

Starting School : A Strengths‐Based Approach Towards Aboriginal And Torres Strait Islander Children, Stephanie Armstrong, Sarah Buckley, Michele Lonsdale, Gina Milgate, Laura Bennetts Kneebone, Louise Cook, Fiona Skelton

Indigenous Education Research

This paper highlights the need for a strengths-based approach to school readiness for Aboriginal and Torres Strait Islander children, in order to recognise the skills, cultural knowledge and understandings they already have when they transition to formal learning. The study, a joint project by the Australian Council for Educational Research (ACER) and the Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA), reviews the literature and uses a strength-based analysis of information from Footprints in Time: The Longitudinal Study of Indigenous Children (LSIC) to examine Aboriginal and Torres Strait Islander children’s abilities and knowledge at 4-6 years of age. …

Enhancing The Quality Of Teaching And Learning In Australian Schools : Submission To The Senate Inquiry On Teaching And Learning (Maximising Our Investment In Australian Schools), Geoff N. Masters

School and system improvement

International research identifies several steps that governments can take to enhance the quality of teaching and learning, and thus levels of student achievement and wellbeing, in primary and secondary schools. The steps are:

1. Restrict and raise the quality of student intakes to teacher education

2. Set and confirm the achievement of minimum standards for registration

3. Recognise and reward the development of specialist knowledge and skill

Review Of Townsville Diocese School Renewal Processes, Tony Mcgruther, Geoff N. Masters

School and system improvement

Townsville Catholic Education Office in Queensland (TCEO) commissioned ACER to review its current processes for school renewal. TCEO seeks to support its school leaders in the design and implementation of self renewing school improvement. This report focuses on the explicit elements of teaching and learning with a particular emphasis on forming recommendations on the evidence base applied to school led reflection. The report is based on data collected from documentation, the views of diocesan leaders, consultants, principals and the current body of research evidence informing school improvement. The recommendations are noted under the common characteristics of highly effective systems identified …

The On Track Survey 2012: The Destinations Of School Leavers In Victoria. Statewide Report, Sheldon Rothman, Catherine Underwood

Vocational, adult and professional education research

The On Track project was initiated in 2003. Since the first large-scale survey in that year, around 385,000 school leavers have participated in the On Track surveys, providing valuable insights into their post-school destinations and pathways. The On Track survey was conducted in April–May 2012 by the Social Research Centre (SRC) in collaboration with the Australian Council for Educational Research (ACER). This involved a short telephone survey of school leavers who had earlier agreed to participate. Separate questionnaires were used for Year 12 or equivalent completers and early school leavers. This report is based on data from telephone surveys of …

International Update (No. 3) June 2012

International Update archive (2010-2012)

Table of contents for this issue: (a) Comparing school learning outcomes internationally; (b) Girls outperform boys in reading in PISA 2009+; (c) Investigating computer and information literacy; (d) Discovering civic knowledge, attitudes and engagement; (e) Expertise and networks from Australia to PNG; (d) Education systems must bridge Indigenous and Western worlds; (e) AHELO fieldwork goes live; (f) Innovative international assessment communities evolve; (g) Quality assurance research and its relevance to China; (h) Professional certification can improve teaching quality.

Changes In Knowledge About And Perception Of Civics And Citizenship Over A Ten-Year Period: Comparing Cived 1999 And Iccs 2009, Wolfram Schulz

Civics and Citizenship Assessment

A larger number of countries involved in the IEA International Civic and Citizenship Education Study (ICCS 2009) had also participated in the previous IEA CIVED study undertaken in 1999. The civic knowledge test included a subset of CIVED items which were used to derive comparable measures of civic content knowledge, one of the subscales reported in CIVED 1999. Some of the ICCS 2009 questionnaire material also included items that were similar or even identical in wording to those used in CIVED but for which some general format changes were applied. This paper discusses the potential for comparing results from both …

Evaluating The Contribution Of Schools To The Development Of Young People As Future Citizens, Bruno Losito, Elisa Caponera, Cristiano Corsini, Paola Mirti

Civics and Citizenship Assessment

This paper will illustrate how the contribution of schools to students’ civic knowledge and attitudes was analyzed in ICCS 2009. It is divided into three parts. The first part illustrates the constructs and variables included in the ICCS instruments (student, teacher and school questionnaires) and their relationship with findings from previous studies about civic and citizenship education and school effectiveness. The second part of the paper presents some major findings for European countries in relation to the influence of school context on student outcomes. While comparing them with the results from other studies, this paper discusses the extent to which …

Measuring Civic Knowledge Among Young Adolescents In The International Civic And Citizenship Education Study, Julian Fraillon, Wolfram Schulz, John Ainley

Civics and Citizenship Assessment

Young people’s acquisition of knowledge about, and their development of and understanding of key concepts in, civics and citizenship is widely seen as an important function of most education systems. From a national perspective, civic knowledge would typically encompass aspects related to the country’s political system and its institutions, rights and responsibilities of citizens, mechanisms of engagement in society as well as general concepts (such as national identity, tolerance or shared values) (Schulz, Ainley, Fraillon, Kerr & Losito, 2010a & 2010b). Depending on the curriculum for civic and citizenship education in a country, it would be expected that these aspects …

Assessing The Intended Participation Of Young Adolescents As Future Citizens: Comparing Results From 26 European Countries, John Ainley, Tim Friedman, David Kerr, Wolfram Schulz

Civics and Citizenship Assessment

Recent developments in the economic and social politics of Europe have led to a quickened interest in the intentions of young people to participate as citizens. This paper focuses on students from 26 European countries who were in Grade 8 in 2009 and who will typically gain formal electoral rights in 2013. It analyzes young adolescents' intentions to participate as citizens through political participation as adults through electoral and active political participation. The paper is based on the IEA International Civic and Citizenship Education Study which investigated the ways in which young people in lower secondary schools were being prepared …

Ib Graduates In Australian Universities : Entry And Outcomes : A Case Study Of Two Institutions : Project Report, Daniel Edwards, Catherine Underwood

Higher education research

This report explores the entry into and progression through university of graduates from the International Baccalaureate (IB) Diploma Programme (DP). It focuses on two Australian universities, used as case studies for exploring the higher education pathways of IB DP graduates. The study has been conducted for the International Baccalaureate, the organisation which oversees the IB DP.

This project responds to one key element of the IB research agenda, which focuses on establishing and expanding empirical evidence related to the postsecondary transitions, academic outcomes and experiences of IB DP graduates in the tertiary education sector. This report is intended to provide …

Innovating A New Future For Learning: Finding Our Path, Valerie Hannon

2009 - 2019 ACER Research Conferences

This keynote paper makes the case that the pursuit of ‘school improvement’ is insufficient to address the challenges facing us if the world is to make available equitable, effective learning systems for all its citizens. A set of drivers of change are explored. It is argued that a ‘split screen’ approach is needed by system leaders and policy makers in which, while school improvement must continue to be pursued, simultaneously a ‘learning ecosystem’ should be created. Such a mutually supportive system would engage a much wider range of partners and players, and would locate learning in a new variety of …

The Neighbourhood Just Got Bigger : Schools And Communities Working Together For Change, Michele Lonsdale, Sharon Clerke, Michelle Anderson

2009 - 2019 ACER Research Conferences

Concurrent Session Block 3