Educational Assessment, Evaluation, and Research Commons^™

Collective Efficacy Tracking Tool. Development Framework 2023, Kerry Elliott, Hilary Hollingsworth

Educational leadership

Collective efficacy has become a prominent feature in educational policy and practice with growing evidence that a strong sense of collective efficacy amongst educators leads to better outcomes for students. With growing interest in the importance of collective efficacy and how to cultivate it, the Collective Efficacy Tracking Tool was created to address challenges associated with understanding, diagnosing, and developing collective efficacy. This Framework describes the development of the Collective Efficacy Tracking Tool, defining collective efficacy in a way that it can be operationalised to leverage improvement efforts. The Framework and accompanying Collective Efficacy Tracking Tool provide a practical resource …

Supporting Schools’ And Teachers’ Use Of Assessment To Inform Learning Of All Students, Ursula Schwantner, Soumaya Maghnouj

Global education monitoring

Issue 5 of the series Using assessment data in education policy and practice: Examples from the Asia-Pacific documents 4 case studies of promising and effective policies or practices on how education systems can better support teachers and schools in effectively integrating assessment practices in the teaching and learning process, to identify and meaningfully address every learner’s needs. In particular, the case studies show how different tools, approaches and policies help promote and reinforce the use of assessment to support more learner-centred and differentiated approaches to teaching and learning. 1. Using the Progressive Achievement approach to develop data-informed schools by Marc …

Reform Challenges In School Education, Geoff N. Masters

School and system improvement

The central thesis of this paper has been that the challenges now confronting schools globally will require fundamental reforms of the external frameworks within which schools work. Today’s challenges to better prepare young people for the future and to ensure that every student learns successfully will not be met by simply expecting teachers to change what they do or by making minor adjustments to current curricula, assessment, examination, reporting and credentialling arrangements. Today’s challenges require deep reforms and a willingness to reimagine—in other words, to ‘transform’ existing learning systems. This is essential because the external frameworks within which teachers and …

Equity And Equality In Learning In Asia-Pacific: What Do Results From Large-Scale Assessments Tell Us? Gender In Focus Policy Brief, Syeda Kashfee Ahmed, Anna Dabrowski, Jacqueline Cheng, Victoria Klimova, Isabella Mccallum

Global education monitoring

This policy brief investigates gender disparities in learning outcomes and the factors contributing to these disparities as identified in large-scale assessment data in Southeast Asia. It additionally provides a set of recommendations aimed at enhancing equitable student learning outcomes while advocating for comprehensive support of gender-based interventions within educational quality initiatives, spanning both systemic and school-level support. The findings presented in this brief are drawn from large-scale assessment (LSA) datasets and secondary analysis of these sources. For more detailed information regarding the large-scale assessments in Southeast Asia, please refer to the appendix section.

Teachers’ Assessment Literacy And Design Competence Framework, Sladana Krstic, Sarah Manlove, Sarah Richardson

2021-2030 ACER Research Conferences

Teachers play a central role in facilitating learning and empowering learners (Pastore & Andrade, 2019). They must possess a high level of assessment literacy to properly fulfil their role of implementing teaching and facilitating students’ learning (Hattie, 2009). Without it, the quality of educational provision can be impacted (Looney et al., 2018). The expectations placed on teachers in relation to their assessment knowledge and skills tend to focus on their ability to design and implement assessment with students, and to use assessment data to inform teaching practices (Brookhart, 2011). Much of the emphasis in the literature is on the mechanical …

Assessing Reading: How Assessment Can Be Used To Target Teaching And Enhance Understanding Of Reading Comprehension, Sandra Knowles

2021-2030 ACER Research Conferences

The skills demonstrated by a proficient reader are not easy to untangle. Current research acknowledges that reading comprehension is a highly complex area of ability, one that needs to be understood as the coordination of several integrated processes. Using example test questions and data, this presentation explores how assessment can help us make sense of reading comprehension in a way that curricula and commonly used teaching strategies cannot. Assessment is evidence that informs us about the skills involved in the reading process, how they relate to each other, and how they develop in complexity. When assessment is understood in this …

Sharing And Securing Learners' Performance Standards Across Schools, Richard Kimbell

2021-2030 ACER Research Conferences

Assessing learners’ performance makes very different demands upon teachers depending on the purpose and the context of the assessment. But common to all assessment is some sense of what ‘quality’ looks like. Most often teachers engage in formative assessments in the classroom, and the familiar standards of the classroom are adequate for this purpose. However if teachers are to undertake external, nationally regulated assessment then some sense of a national standard of quality is required. But there are very limited mechanisms by which teachers can acquire this understanding, so they use their best judgement, and standards vary from school to …

An Innovative Method For Teachers To Formatively Assess Writing Online, Dr Sandy Heldsinger, Stephen Humphry

2021-2030 ACER Research Conferences

Assessment is an integral component of effective teaching and a teacher’s professional judgement influences all routine aspects of their work. In the last 20 years, there has been considerable work internationally to support teachers in using assessment to improve student learning. However, there is a pressing issue that impedes teacher professional judgement being exploited to its full potential. The issue relates to teacher assessments in the context of extended performances such as essays and arises from the complexity of obtaining reliable or consistent teacher assessments of students’ work. Literature published in the United States, England and Australia details evidence of …

Using Pat Data To Inform Teaching And Learning, Marc Kralj, Rachel Felgate, Shani Sniedze, Caithlin Power, Grant Barclay, Darren Leech

2021-2030 ACER Research Conferences

ACER works with thousands of Australian schools to set up Progressive Achievement Test (PAT) functionality, assign and administer tests, and to assist teachers to understand and interrogate the resulting test data and act upon findings. This session provides participants with an opportunity to hear about a school’s journey, and discuss this with the ACER team and our PAT school guest. ACER researchers have evaluated and documented assessment implementation and professional development opportunities, observing this school’s ability to track student and educator progress over time, which is a critical element in ACER’s Progressive Achievement approach. ACER’s educational consultants will demonstrate how …

Reimagining Classroom Assessment And Feedback To Meet Learner Needs, Fabienne Van Der Kleij

2021-2030 ACER Research Conferences

The power of classroom assessment and feedback to improve student learning outcomes has long been recognised. Yet, decades of research have yielded disappointing and often conflicting outcomes. This presentation challenges traditional conceptions of classroom assessment and feedback as teacher-driven practices. To meet learner needs better, it proposes a student-centred perspective in which students are active and have agency. By drawing on an extensive study of feedback reviews, this presentation illustrates how conceptualisations of feedback have changed over recent decades. This paper provides key insights into how classroom assessment and feedback practices can be changed in ways that are sustainable, afford …

Reimagining The Purpose Of Assessment, Geoff N. Masters

2021-2030 ACER Research Conferences

Assessment is commonly understood as the process of judging how well students have learnt what they have been taught. It comes at the end of a sequence that begins with a curriculum or course syllabus. Teachers are expected to deliver this body of specified content, students are expected to learn it, and assessment is the process of judging and grading students on how well they have learnt what teachers have taught. This is a common view of assessment among students, parents and many teachers. I will argue in this presentation that this traditional understanding of what it means to learn …

The Swans/Ables Project: A Set Of Resources Developed Collaboratively With Teachers To Support The Teaching And Learning Of Students With Additional Learning Needs, Toshiko Kamei

2021-2030 ACER Research Conferences

The inclusion of students with additional learning needs in schooling is part of policy and practice in Australia. However, it has been well documented that teachers lack the resources and training to meaningfully include students with additional learning needs in the full range of learning in their classrooms. The SWANs (Students with Additional Needs) program of work aimed to fill this gap through developing assessments based on learning progressions to provide targeted information to support the teaching and learning of all students, including students with additional learning needs. The development and implementation of the SWANs/ABLES suite of resources illustrates how …

How Might We Identify And Measure Learning Progression In History?, Louise Zarmati

2021-2030 ACER Research Conferences

In this session, Dr Zarmati will share her research on efforts to map and describe progress in the learning area of History.

Learning progression is a continuum that measures advances in learning by tracking development from early learning to more sophisticated levels of mastery. Mathematics relies on an understanding of empirical knowledge and concepts in a hierarchical sequence; students need to understand (or master) one mathematical concept before they can proceed to the next. In comparison, progress of understanding in history is not necessarily hierarchical because it is based on mastery of concepts and skills rather than historical knowledge, which …

Leading System Transformation: A Work In Progress, Greg Whitby, Maura Manning, Gavin Hays

2021-2030 ACER Research Conferences

Internationally, the COVID-19 pandemic has profoundly disrupted the education sector. While NSW has avoided the longer periods of remote learning that our colleagues in Victoria and other countries have experienced, we have nonetheless been provoked to reflect on the nature of schooling and the systemic support we provide to transform the learning of each student and enrich the professional lives of staff within our Catholic learning community. At Catholic Education Diocese of Parramatta (CEDP), a key pillar of our approach is to create conditions that enable everyone to be a leader. Following the initial lockdown period in 2020 when students …

Interpreting Learning Progress Using Assessment Scores: What Is There To Gain?, Nathan Zoanetti

2021-2030 ACER Research Conferences

Using assessment scores to quantify gains and growth trajectories for individuals and groups can provide a valuable lens on learning progress for all students. This paper summarises some commonly observed patterns of progress and illustrates these using data from ACER’s Progressive Achievement Test (PAT) assessments. While growth trajectory measurement requires scores for the same individuals over at least three but preferably more occasions, scores from only two occasions are naturally more readily available. The difference between two successive scores is usually referred to as gain. Some common approaches and pitfalls when interpreting individual student gain data are illustrated. It is …

Using Assessment Data To Improve Equity: How Teachers Use Insights From The Scottish National Standardised Assessments, Sarah Richardson, Sladana Krstic

2021-2030 ACER Research Conferences

Evidence-based decision-making is regarded as an important indicator of quality in schools around the world. Using data gathered from assessments, in conjunction with other insights, can help school leaders and teachers better meet the needs of learners. In schools that cater to disadvantaged learners, using data to design targeted interventions plays an important role in improving equity. In this paper we report on a study with five schools in Scotland. All schools had learner cohorts characterised by multiple layers of disadvantage. Informed by the theoretical underpinnings of sensemaking theory, we investigated how teachers and school leaders used data from the …

Identifying And Monitoring Progress In Collaboration Skills, Claire Scoular

2021-2030 ACER Research Conferences

The nature of skills such as collaboration is complex, particularly given that there are internal processes at play. Inferences need to be made to interpret explicit behaviours observed from intentionally designed assessment tasks. This paper centres on the approach to develop hypotheses of skill development into validated learning progressions using assessment data. Understanding a skill from a growth perspective is essential for the effective teaching and development of the skill. The application of Item Response Theory (IRT) allows the interpretation of assessment data as levels of proficiency that we can use to map or monitor progress in collaborative skills.

Evidencing Creativity And Curiosity In Ib Schools, Sarah Richardson, Sladana Krstic

2021-2030 ACER Research Conferences

There is growing recognition of the importance of learners gaining transversal or 21st-century attributes in order to thrive in the contemporary world. This poses a number of challenges for educators. First, to what extent are transversal attributes innate, or do they include a combination of traits and skills? Second, what can teachers do to help nurture these attributes in learners? Third, how can the existence or strengthening of attributes be recognised? In this paper, we draw on work that we are doing for the International Baccalaureate Organisation to define conceptual frameworks for creativity and curiosity. Our goal is to enable …

Applying Empirical Learning Progressions For A Holistic Approach To Evidence-Based Education: Swans/Ables, Emily White

2021-2030 ACER Research Conferences

Learning progressions have become an increasing topic of interest for researchers, educational organisations and schools as they can describe the expected pathway of learning within a content area to allow for targeted teaching and learning at all levels of ability. However, there is substantial variation in how learning progressions are developed and to what extent teachers can use them to inform their practices. The ABLES/SWANS tools (Students with Additional Needs/Abilities Based Learning and Education Support) are an example of how an empirical learning progression can be applied to support teachers’ ability to not only target teaching to a student’s zone …

Learning Progressions As Models And Tools For Supporting Classroom Assessment, Alicia C. Alonzo

2021-2030 ACER Research Conferences

Like all models, learning progressions (LPs) provide simplified representations of complex phenomena. One key simplification is the characterisation of student thinking in terms of levels. This characterisation is both essential for large-scale applications, such as informing standards, but potentially problematic for smaller-scale applications. In this paper, I describe a program of research designed to explore the smaller-scale use of LPs as supports for teacher classroom assessment practices in light of this simplification. Based on this research, I conclude that LP levels may serve as a generative heuristic, particularly when teachers are engaged with evidence of the limitations of LP levels …

Karmel Oration: Excellent Progress For All: A Function Of Year-Level Curriculum Or Evidenced-Based Learning Progressions?, Dianne Siemon

2021-2030 ACER Research Conferences

Excellent progress for all students is an ambitious but necessary goal if we are to improve the life choices of all students. At the moment, we are not serving all our students well despite the best efforts of teachers. We need to look further afield to the curriculum and assessment regimes that drive current practice. Grouping students by ability and offering a watered-down curriculum for some is not the answer. Evidenced-based learning progressions that point to what is important in ensuring all students build a deep, well-connected understanding of mathematics over time is what is needed to support reform at …

This Time Without ‘Feeling’: Children’S Intuitive Theories Of Art As A Logical Basis For Learning Progression In Visual Arts, Karen Maras

2021-2030 ACER Research Conferences

Learning in Visual Arts has traditionally been framed as an experiential process in which feeling and intuition complement the development of aesthetic knowledge. However, while art can be about feelings and processes that develop students’ expressive capacities, the complexity of art understanding and thinking extends beyond this narrow common-sense assumption. I argue that this assumption, which is represented in the Australian Curriculum: The Arts (ACARA, 2015), and even more firmly resonates in recent proposals for the revision of this curriculum (ACARA, 2021), obfuscates the conceptual and theoretical bases on which students make progress in art understanding. This paper examines the …

How Education Gets In The Way Of Learning, Geoff N. Masters

2021-2030 ACER Research Conferences

The formal structures and processes of school education – including the organisation of the school curriculum, processes for assessing student learning, methods of reporting performance, and the uses to which student results are put – are often inconsistent with what is now known about the best ways to promote human learning. Rather than being designed to maximise every student’s learning, these structures and processes often reflect 20th century priorities, including the use of school education to sort and select students into different education and training destinations, and future careers. This sorting function of schooling is becoming increasingly irrelevant in knowledge …

Research Conference 2021: Excellent Progress For Every Student: Proceedings And Program, Australian Council For Educational Research

2021-2030 ACER Research Conferences

The focus of the 2021 Australian Council for Educational Research (ACER) Research Conference is on evidence-based strategies for ensuring that every student makes excellent ongoing progress in their learning. This is an important topic because many students in our schools do not make good, steady progress. Some slip behind and fall increasingly behind the longer they are in school. By the middle years of school, this contributes to significant levels of student disengagement. ACER has invited a number of leading educational researchers to join us to share the findings of their research relevant to this topic. An important conclusion of …

Peer Observation: A Collaborative And Mutually Rewarding Activity, Deepali Dharmaraj

Teacher India

Peer observation is an effective tool for improving teaching instructions and career development, says Deepali Dharmaraj.

Enhancing The National Assessment System To Improve Learning Outcomes In Bhutan: Case Study 2020-2021, Anu Radha Sharma, Anannya Chakraborty, Arjun Kumar Gurung

Assessment and Reporting

To measure the efficacy of school sector interventions, Bhutan Council for School Examinations and Assessment (BCSEA) partnered with Australian Council for Educational Research (India) to strengthen its National Education Assessment (NEA) for obtaining evidence on learning, monitoring the progress of learning outcomes, understanding associations between context and achievement, and differences in achievement between groups. Eventually, the assessment reform will support measuring learning at the key stages of grades 3, 6, and 9. This case study report presents an overview of this project.

Nurturing Wonder And Igniting Passion. Designs For A New School Curriculum: Nsw Curriculum Review, Geoff N. Masters

NSW Curriculum Review

This Review of the New South Wales school curriculum has concluded that change is required. The changes recommended by the Review are far-reaching and amount to the introduction of a new curriculum for NSW schools, from Kindergarten to Year 12. The syllabuses of this new curriculum are designed to ensure every student learns with understanding, builds skills in applying knowledge, and makes excellent ongoing progress in their learning. The goal is to provide every student, in each phase of their learning, with strong foundations for what comes next. Detailed planning, piloting and implementation of the new curriculum will be required …

Creative Thinking: Skill Development Framework, Dara Ramalingam, Prue Anderson, Daniel Duckworth, Claire Scoular, Jonathan Heard

Assessment and Reporting

The Australian Council for Educational Research (ACER) has reviewed the extensive literature on creative thinking and developed both a definition and a framework that synthesise and harmonise existing theory and research on creative thinking. This framework has been developed to address the challenges associated with teaching and assessing creative thinking. It outlines creative thinking processes along prescribed strands and aspects informed by a sound evidentiary basis. The aspects contained within the framework are designed to provide foci for teaching and the basis of assessment. ACER’s creative thinking framework identifies key factors that underpin the development of creative thinking with a …

Critical Thinking: Skill Development Framework, Jonathan Heard, Claire Scoular, Daniel Duckworth, Dara Ramalingam, Ian Teo

Assessment and Reporting

The ACER critical thinking framework has been developed to address the challenges associated with teaching and assessing critical thinking. While there are many definitions of the skill, which are outlined in the first part of this document, few provide a means to operationalise critical thinking in the classroom. This framework outlines critical thinking processes along prescribed strands and aspects informed by a sound evidentiary basis. The aspects contained within the framework are designed to provide foci for teaching and the basis of assessment. This paper is divided into three parts. Part 1 explores the development of critical thinking as an …

Collaboration: Skill Development Framework, Claire Scoular, Daniel Duckworth, Jonathan Heard, Dara Ramalingam

Assessment and Reporting

This skill development framework has been developed to address the challenges associated with teaching and assessing collaboration. While there are many definitions of the skill, few provide a means to operationalise collaboration in the classroom. This framework is designed to synthesise and harmonise existing theory and research on collaboration to provide a holistic perspective. It outlines collaboration processes along prescribed strands and aspects that are informed by a sound evidentiary basis. The aspects contained within the framework are designed to provide foci for teaching and the basis of assessment.