Educational Assessment, Evaluation, and Research Commons^™

Collaboration In East Africa: A Contextualised Approach To Defining The Construct, Claire Scoular, David Alelah Otieno

Assessment and Reporting

Collaboration has been highlighted internationally as a key skill for learning, working, and living in the twenty-first century. However, to teach it well, enhance its performance, and measure its growth, it is essential to have a clear and consistent definition of the skill. There are a number of frameworks that describe collaboration in a way that is meaningful to learning and growth. Despite some differences across frameworks, it is clear there is a common core set of contributing subskills. This suggests that collaboration is of global interest, and that there are components that transcend national or cultural specificities. Notwithstanding, definitions …

National Assessment Program: Ict Literacy 2022: Public Report, Tim Friedman, Kate O'Malley, Zachary Watts, Kristy Osborne, Louise Ockwell, Laila Helou

ICT - Digital Literacy

This public report documents the findings of the sixth National Assessment Program ICT Literacy (NAP–ICT Literacy) assessment cycle. In reporting national key performance measures (KPMs) of Australian students’ ICT literacy, the NAP–ICT Literacy assessment provides a way to monitor progress towards the Alice Springs (Mparntwe) Education Goals for Young Australians. Goal 2 of the Alice Springs (Mparntwe) Education Declaration is that “all young Australians become confident and creative individuals, successful lifelong learners, and active and informed members of the community” (Education Council 2019, p. 6). The elaboration of this goal highlights the importance of young Australians’ digital and ICT literacy …

The Contribution Of Learning Trajectories To Enacting The Early Years Learning Framework V2.0, Caroline Cohrssen

2021-2030 ACER Research Conferences

The Early Years Learning Framework for Australia v2.0 (EYLF) guides pedagogy and practice with children aged from birth to 5 years and states that ‘over time, children engage with increasingly complex ideas’. With 5 learning outcomes and 8 principles of practice, this requires educators to be highly skilled in facilitating children’s engagement with increasingly complex ideas. It also assumes that all educators recognise children’s demonstrations of understanding, and know what knowledge (or capabilities) likely preceded this understanding, and what comes next. As a framework, this specific information is missing from the EYLF. Learning trajectories may assist educators to recognise demonstrations …

Mathematical Mindsets: Fostering Student Engagement And Positive Mindsets Through The Use Of Challenging Tasks, Alison Hall

2021-2030 ACER Research Conferences

This paper explores the planned use of challenging mathematical tasks. These tasks provide the opportunity for students to improve mathematical thinking by working on problems that they do not yet know how to answer. This research involved a heterogeneous class of year 3 students from a Catholic parish primary school in the northern suburbs of Melbourne. A rubric was also developed to use in conjunction with these tasks, to support discussions with students, broaden their strategies in finding solutions and thereby improve their conceptual understanding. These pedagogical approaches were found to support the improvement of both students’ conceptual understanding in …

Assessment Is Coming And The Early Childhood Sector Must Lead The Way, Dan Cloney

2021-2030 ACER Research Conferences

Assessment is a core component of quality early childhood practice. It is explicitly highlighted in the new Early Years Leanring Framework V2.0 and is a standard within Quality Area 1 of the National Quality Standard. In everyday early childhood education and care (ECEC) settings, and in initial teacher education, assessment is often limited to observational and narrative-driven approaches. Recent reviews of the literature highlight that there are few other assessment tools readily available to educators. What assessment looks like in early childhood is changing. The Commonwealth, as part of the Preschool Reform Funding Agreement, is developing, trialling, and implementing a …

Partnerships In Early Childhood Education: Engaging Families In Professional Conversations, Simone Griggs, Greta Rollo

2021-2030 ACER Research Conferences

Effective monitoring of learning and development involves integrating multiple perspectives from a child’s parents, carers, family, and their educators. A collaborative approach between educators and families not only recognises the family’s critical importance in their child’s life but it has also been shown to benefit children with learning difficulties, delays and behavioural concerns (Parenting Research Centre, 2012; Prior et al., 2011). Efforts made towards the integration of multiple perspectives can also support stronger inclusion of diverse families into communities of early childhood education and care. However, discussions with parents and carers about issues regarding their children’s development can be a …

Developing A Learning Progression For Teachers To Support Effective Thinking And Self-Regulation, Hannah Campos Remon, Adam Kuss

2021-2030 ACER Research Conferences

Brisbane Grammar School has undertaken four years of systematic research to develop a learning progression to support teachers in the development of their students’ capacity for learning, including self-regulation. The learning progression offers a comprehensive, learning-focused tool for teachers to consider how their students are developing as learners over time. Whilst it is not intended to be prescriptive or exhaustive, or to produce a grade for ‘learning’, it offers a purposeful sampling of important components of effective thinking.

Transdisciplinary Stem Enactment: An Exploratory Case Study In The Queensland Context, Kristie Schulz

2021-2030 ACER Research Conferences

Enacting STEM education in Australian schools is an ambitious task, in a climate of unclear definitions and little implementation advice. Should STEM education simply refer to an umbrella-term of subjects that Australian students need to improve in, or could a cross-disciplinary pedagogy that engages students in authentic problem-solving be realised? This poster outlines a project in which Parklands Christian College designed and enacted a transdisciplinary student-centred ideology of STEM education in 2017. The program has continually evolved using a transdisciplinary pedagogical framework.

School Improvement Tool, Australian Council For Educational Research (Acer)

School and system improvement

Research shows the powerful impact that school leadership teams can have in improving the quality of teaching and learning. Effective leaders create cultures of high expectations, provide clarity about what teachers are to teach and students are to learn, establish strong professional learning communities, and lead ongoing school-wide efforts to improve teaching practices. The School Improvement Tool describes the practices of highly effective schools and school leaders. The SIT assists schools to review and reflect on their strategies to improve the quality of classroom teaching and learning. It supports school-wide conversations – including with families, school governing bodies, local communities, …

Teachers’ Assessment Literacy And Design Competence Framework, Sladana Krstic, Sarah Manlove, Sarah Richardson

2021-2030 ACER Research Conferences

Teachers play a central role in facilitating learning and empowering learners (Pastore & Andrade, 2019). They must possess a high level of assessment literacy to properly fulfil their role of implementing teaching and facilitating students’ learning (Hattie, 2009). Without it, the quality of educational provision can be impacted (Looney et al., 2018). The expectations placed on teachers in relation to their assessment knowledge and skills tend to focus on their ability to design and implement assessment with students, and to use assessment data to inform teaching practices (Brookhart, 2011). Much of the emphasis in the literature is on the mechanical …

Assessing Reading: How Assessment Can Be Used To Target Teaching And Enhance Understanding Of Reading Comprehension, Sandra Knowles

2021-2030 ACER Research Conferences

The skills demonstrated by a proficient reader are not easy to untangle. Current research acknowledges that reading comprehension is a highly complex area of ability, one that needs to be understood as the coordination of several integrated processes. Using example test questions and data, this presentation explores how assessment can help us make sense of reading comprehension in a way that curricula and commonly used teaching strategies cannot. Assessment is evidence that informs us about the skills involved in the reading process, how they relate to each other, and how they develop in complexity. When assessment is understood in this …

Assessment Moderation: Is It Fit For Purpose?, Lenore Adie

2021-2030 ACER Research Conferences

This presentation outlines the different practices and purposes of assessment moderation and analyses the benefits and issues of these. Two projects are presented; each seeks to improve teacher judgement. Both attempt to redefine what we mean by moderation and how moderation can be conducted to reach diverse and dispersed groups. The first draws on a current Australian Research Council project that is exploring the development of scaled exemplars, the use of commentaries of judgement decisions, and the use of digital platforms to support teachers grading student work. The second draws on the work of the Graduate Teacher Performance Assessment (GTPA) …

The Transformational Change Of Adaptive Assessment, Jarrod Hingston, Toby Newton

2021-2030 ACER Research Conferences

Organisational change theory often discusses the differences between adaptive change and transformational change; adaptive change being the process of incremental shifts that are intended to drive improvement, while transformational change requires major alterations in processes. In 2019, ACER embarked on a mission to develop a new adaptive testing and reporting system, with the goal of delivering an assessment that would improve the precision of results for every student. While the intention was to introduce an adaptive change in the assessments, the project truly transformed the way that ACER delivers assessments and reports results, a result recognised by the project being …

Sharing And Securing Learners' Performance Standards Across Schools, Richard Kimbell

2021-2030 ACER Research Conferences

Assessing learners’ performance makes very different demands upon teachers depending on the purpose and the context of the assessment. But common to all assessment is some sense of what ‘quality’ looks like. Most often teachers engage in formative assessments in the classroom, and the familiar standards of the classroom are adequate for this purpose. However if teachers are to undertake external, nationally regulated assessment then some sense of a national standard of quality is required. But there are very limited mechanisms by which teachers can acquire this understanding, so they use their best judgement, and standards vary from school to …

An Innovative Method For Teachers To Formatively Assess Writing Online, Dr Sandy Heldsinger, Stephen Humphry

2021-2030 ACER Research Conferences

Assessment is an integral component of effective teaching and a teacher’s professional judgement influences all routine aspects of their work. In the last 20 years, there has been considerable work internationally to support teachers in using assessment to improve student learning. However, there is a pressing issue that impedes teacher professional judgement being exploited to its full potential. The issue relates to teacher assessments in the context of extended performances such as essays and arises from the complexity of obtaining reliable or consistent teacher assessments of students’ work. Literature published in the United States, England and Australia details evidence of …

Using Pat Data To Inform Teaching And Learning, Marc Kralj, Rachel Felgate, Shani Sniedze, Caithlin Power, Grant Barclay, Darren Leech

2021-2030 ACER Research Conferences

ACER works with thousands of Australian schools to set up Progressive Achievement Test (PAT) functionality, assign and administer tests, and to assist teachers to understand and interrogate the resulting test data and act upon findings. This session provides participants with an opportunity to hear about a school’s journey, and discuss this with the ACER team and our PAT school guest. ACER researchers have evaluated and documented assessment implementation and professional development opportunities, observing this school’s ability to track student and educator progress over time, which is a critical element in ACER’s Progressive Achievement approach. ACER’s educational consultants will demonstrate how …

Reimagining Classroom Assessment And Feedback To Meet Learner Needs, Fabienne Van Der Kleij

2021-2030 ACER Research Conferences

The power of classroom assessment and feedback to improve student learning outcomes has long been recognised. Yet, decades of research have yielded disappointing and often conflicting outcomes. This presentation challenges traditional conceptions of classroom assessment and feedback as teacher-driven practices. To meet learner needs better, it proposes a student-centred perspective in which students are active and have agency. By drawing on an extensive study of feedback reviews, this presentation illustrates how conceptualisations of feedback have changed over recent decades. This paper provides key insights into how classroom assessment and feedback practices can be changed in ways that are sustainable, afford …

Acer Models Of Writing: Changing The Assessment Mindset, Juliette Mendelovits, Judy Nixon

2021-2030 ACER Research Conferences

This presentation outlines different models of writing assessment that have a proven track record in various programs delivered by ACER. These models include holistic scoring, and partial analytical scoring on a number of criteria. The writing prompts for these models assume extended pieces of writing. The presentation will also introduce a new and alternative model of writing assessment that ACER has implemented in several large-scale assessments. This model is built on shorter pieces of writing from each student, designed to give more accurate and diverse insights into students’ proficiency in writing in a range of text types, within the constraints …

Reimagining The Purpose Of Assessment, Geoff N. Masters

2021-2030 ACER Research Conferences

Assessment is commonly understood as the process of judging how well students have learnt what they have been taught. It comes at the end of a sequence that begins with a curriculum or course syllabus. Teachers are expected to deliver this body of specified content, students are expected to learn it, and assessment is the process of judging and grading students on how well they have learnt what teachers have taught. This is a common view of assessment among students, parents and many teachers. I will argue in this presentation that this traditional understanding of what it means to learn …

Research Conference 2022: Reimagining Assessment: Proceedings And Program, Kylie Burns (Ed.)

2021-2030 ACER Research Conferences

The focus of this year’s Research Conference is on the use of assessment to support improved teaching and learning. The conference is titled ‘reimagining assessment’ because we believe there is a need to transform the essential purposes of educational assessment to provide better information about the deep conceptual learning, skills, competencies, and personal attributes that teachers and schools now have as objectives for student learning and development. Reimagined assessments must now be focused on monitoring learning across this broader range of intended outcomes and provide quality information about the points individuals have reached in their long-term development.

Development Of A Transcript To Record Learner Creativity And Curiosity, Prue Anderson, Toby Carslake, Anit Cherian, Bethany Davies, Laura Good, Jonathan Heard, Sladana Krstic, Brendan Mcginley, Dara Ramalingam, Sarah Richardson, Claire Scoular, Shani Sniedze, Jennifer Star

Assessment and Reporting

Funded with generous support by the Jacobs Foundation, the ultimate goal of this project was to develop transcripts to track learner progress in the domains of creativity and curiosity. To support this overarching goal, the research team sought to define creativity and curiosity in language that would resonate with learners and teachers and that would be appropriate across numerous cultural settings. The result of the project is a series of prototype materials and resources, specifically: literature reviews, frameworks, enabling environment summaries, reflective quizzes and transcripts. Based on insights from the literature reviews, the researchers developed frameworks to define the constructs …

Reporting Student Progress: What Might It Look Like?, Hilary Hollingsworth, Jonathan Heard, Anthony Hockey, Tegan Knuckey

2021-2030 ACER Research Conferences

The Communicating Student Learning Progress review produced by ACER in 2019 set out recommendations for schools and systems to improve the way schools report on student learning, in particular learning progress. Two case study schools from Victoria – a Catholic primary school and government secondary school – discuss changes they’ve made to their student reporting processes, in response to the review’s recommendations. Further research is recommended into how schools are rethinking reporting to engage students and parents in monitoring learning growth.

The Swans/Ables Project: A Set Of Resources Developed Collaboratively With Teachers To Support The Teaching And Learning Of Students With Additional Learning Needs, Toshiko Kamei

2021-2030 ACER Research Conferences

The inclusion of students with additional learning needs in schooling is part of policy and practice in Australia. However, it has been well documented that teachers lack the resources and training to meaningfully include students with additional learning needs in the full range of learning in their classrooms. The SWANs (Students with Additional Needs) program of work aimed to fill this gap through developing assessments based on learning progressions to provide targeted information to support the teaching and learning of all students, including students with additional learning needs. The development and implementation of the SWANs/ABLES suite of resources illustrates how …

How Might We Identify And Measure Learning Progression In History?, Louise Zarmati

2021-2030 ACER Research Conferences

In this session, Dr Zarmati will share her research on efforts to map and describe progress in the learning area of History.

Learning progression is a continuum that measures advances in learning by tracking development from early learning to more sophisticated levels of mastery. Mathematics relies on an understanding of empirical knowledge and concepts in a hierarchical sequence; students need to understand (or master) one mathematical concept before they can proceed to the next. In comparison, progress of understanding in history is not necessarily hierarchical because it is based on mastery of concepts and skills rather than historical knowledge, which …

Supporting Science Teaching Practice With Learning Progressions, Erin Furtak

2021-2030 ACER Research Conferences

Learning progressions are often used as foundations for curriculum and assessment. At the same time, as representations of the development of student ideas and practices, they can also serve as maps to support teachers during instruction. This paper describes a program of research in which my colleagues and I have investigated how learning progressions can support high school science teachers in cycles of co-designing formative assessments.

Rethinking Measurement For Accountable Assessment, Mark Wilson

2021-2030 ACER Research Conferences

The underlying model for most formal educational measurement (e.g. standardised tests) is based on a very simple model: the student takes a test (possibly alongside other students). The complications of there being an instructional plan, actual instruction, interpretation of the outcome, and formulation of next steps, are all bypassed in considering how to model the process of measurement. There are some standard exceptions, of course: a pre-test/post-test context will involve two measurements, and attention to gain score, or similar. However, if we wish to design measurement to hold to Lehrer’s (2021) definition of ‘accountable assessment’ – as ‘actionable information for …

Making Excellent Progress In Early Reading: How Can The Identification Of Essential Skills And Concepts Help?, Dara Ramalingam, Prue Anderson, Sandra Knowles, Danielle Anzai, Greta Rollo

2021-2030 ACER Research Conferences

The ability to read and understand text is fundamental to full participation in modern adult life (Olson, 1977; Elwert, 2001). It is essential to educational progress across domains, but increased literacy levels are also linked to positive outcomes in terms of employment and health. Given its critical role both in the facilitation of learning in all domains, and in many aspects of life beyond school, it is imperative that we give students the best possible chance to develop their reading skills. This paper uses early reading as a case study for examining how the identification and explication of essential skills …

Leading System Transformation: A Work In Progress, Greg Whitby, Maura Manning, Gavin Hays

2021-2030 ACER Research Conferences

Internationally, the COVID-19 pandemic has profoundly disrupted the education sector. While NSW has avoided the longer periods of remote learning that our colleagues in Victoria and other countries have experienced, we have nonetheless been provoked to reflect on the nature of schooling and the systemic support we provide to transform the learning of each student and enrich the professional lives of staff within our Catholic learning community. At Catholic Education Diocese of Parramatta (CEDP), a key pillar of our approach is to create conditions that enable everyone to be a leader. Following the initial lockdown period in 2020 when students …

Interpreting Learning Progress Using Assessment Scores: What Is There To Gain?, Nathan Zoanetti

2021-2030 ACER Research Conferences

Using assessment scores to quantify gains and growth trajectories for individuals and groups can provide a valuable lens on learning progress for all students. This paper summarises some commonly observed patterns of progress and illustrates these using data from ACER’s Progressive Achievement Test (PAT) assessments. While growth trajectory measurement requires scores for the same individuals over at least three but preferably more occasions, scores from only two occasions are naturally more readily available. The difference between two successive scores is usually referred to as gain. Some common approaches and pitfalls when interpreting individual student gain data are illustrated. It is …

Using Assessment Data To Improve Equity: How Teachers Use Insights From The Scottish National Standardised Assessments, Sarah Richardson, Sladana Krstic

2021-2030 ACER Research Conferences

Evidence-based decision-making is regarded as an important indicator of quality in schools around the world. Using data gathered from assessments, in conjunction with other insights, can help school leaders and teachers better meet the needs of learners. In schools that cater to disadvantaged learners, using data to design targeted interventions plays an important role in improving equity. In this paper we report on a study with five schools in Scotland. All schools had learner cohorts characterised by multiple layers of disadvantage. Informed by the theoretical underpinnings of sensemaking theory, we investigated how teachers and school leaders used data from the …