Education Commons^™

Pairwise Comparison Method: Concept Note, Australian Council For Educational Research

Global education monitoring

The paper sets out several approaches, both statistical and non-statistical, by which jurisdictions may choose to link their assessments to the global standards for reporting. One of these, the Pairwise Comparison Method (PCM), was first set out in Lazendic (2019). This concept note provides further details on the PCM and the work that has taken place since 2019 to enable the approach to be operationalised. The PCM relies on the Learning Progression Scales (LPSs) for reading and mathematics, developed by the Australian Council for Educational Research (ACER). A full description of the development of the scales is provided in the …

Assessment Moderation: Is It Fit For Purpose?, Lenore Adie

2021-2030 ACER Research Conferences

This presentation outlines the different practices and purposes of assessment moderation and analyses the benefits and issues of these. Two projects are presented; each seeks to improve teacher judgement. Both attempt to redefine what we mean by moderation and how moderation can be conducted to reach diverse and dispersed groups. The first draws on a current Australian Research Council project that is exploring the development of scaled exemplars, the use of commentaries of judgement decisions, and the use of digital platforms to support teachers grading student work. The second draws on the work of the Graduate Teacher Performance Assessment (GTPA) …

Sharing And Securing Learners' Performance Standards Across Schools, Richard Kimbell

2021-2030 ACER Research Conferences

Assessing learners’ performance makes very different demands upon teachers depending on the purpose and the context of the assessment. But common to all assessment is some sense of what ‘quality’ looks like. Most often teachers engage in formative assessments in the classroom, and the familiar standards of the classroom are adequate for this purpose. However if teachers are to undertake external, nationally regulated assessment then some sense of a national standard of quality is required. But there are very limited mechanisms by which teachers can acquire this understanding, so they use their best judgement, and standards vary from school to …

Research Conference 2022: Reimagining Assessment: Proceedings And Program, Kylie Burns (Ed.)

2021-2030 ACER Research Conferences

The focus of this year’s Research Conference is on the use of assessment to support improved teaching and learning. The conference is titled ‘reimagining assessment’ because we believe there is a need to transform the essential purposes of educational assessment to provide better information about the deep conceptual learning, skills, competencies, and personal attributes that teachers and schools now have as objectives for student learning and development. Reimagined assessments must now be focused on monitoring learning across this broader range of intended outcomes and provide quality information about the points individuals have reached in their long-term development.

Building A World-Class Learning System, Geoff N. Masters

School and system improvement

This study sought to understand how five jurisdictions – British Columbia, Estonia, Finland, Hong Kong and South Korea, which have all performed unusually well in international achievement surveys over the past two decades – are approaching school education and its transformation. Geoff Masters describes key aspects of the learning systems these jurisdictions have established, comments on how they are now redesigning their learning systems for the future, and discusses insights into what may be required for any jurisdiction to perform well on measures of the kind currently used in international surveys.

Teachers’ Perceptions And Experiences During An Annual Performance And Development Cycle, Kerry Elliott, John Hattie, Lorraine Graham

Australian Journal of Teacher Education

In this study, the perceptions and experiences of eighteen teachers across three primary schools in Victoria, Australia, were examined as they participated in an annual performance and development cycle, guided by the Australian Teacher Performance and Development Framework. The study sought to investigate teachers’ experiences and perceptions of the cycle to understand the aspects perceived as valuable to these teachers. Semi-structured interviews and focus groups were held with teachers at the beginning and end of each school’s annual performance and development cycle, and responses were thematically analysed. Findings suggest that school leaders perform a critical role in the …

Rethinking Measurement For Accountable Assessment, Mark Wilson

2021-2030 ACER Research Conferences

The underlying model for most formal educational measurement (e.g. standardised tests) is based on a very simple model: the student takes a test (possibly alongside other students). The complications of there being an instructional plan, actual instruction, interpretation of the outcome, and formulation of next steps, are all bypassed in considering how to model the process of measurement. There are some standard exceptions, of course: a pre-test/post-test context will involve two measurements, and attention to gain score, or similar. However, if we wish to design measurement to hold to Lehrer’s (2021) definition of ‘accountable assessment’ – as ‘actionable information for …

Standard Specific Student Achievement Scores And The Impact On Teacher Incorporation Of Standards, Knowledge Of Standards, And Student Achievement, Jeffrey Burgess

Dissertations, Theses, and Projects

There has been a steady focus in education on standards, assessment, and accountability. The purpose of this quantitative study was to find if research based PLCs focusing on standard specific student achievement data impacted teacher knowledge of standards, incorporation of standards, and student achievement. The research was guided by two research questions. Within the context of PLCs (data-based teams) utilizing standard specific student achievement data: 1) What impact, if any, was the use of PLCs on the teachers’ knowledge and incorporation of standards into the classroom setting? 2) Did student achievement on standardized tests improve as the result of the …

Understanding How Experienced World History Teachers In Minnesota Choose Course Content: A Mixed Methods Study, Kathleen Ferrero

All Graduate Theses, Dissertations, and Other Capstone Projects

Education in the United States has consistently utilized tests as well as a relatively standard curriculum to educate the youth. However, the introduction of the No Child Left Behind Act and the subsequent implementation of the Common Core State Standards Initiative, has created an educational environment focused largely on math and literacy skills. This mixed methods study discovered the primary factors experienced world history teachers in Minnesota utilize to balance historical thinking skills and the inclusion of current events against national and statewide curricular standards. Five experienced teachers were interviewed, and their responses coded using qualitative methods. These themes informed …

Validation Of The Myanmar Teacher Competency Standards Framework (Tcsf) Final Report, Anna Dabrowski, Jeaniene Spink

Teacher education

This final report provides background to the four domains of the Myanmar Teacher Competency Standards Framework (TCSF), and the validation study design, stages and methods which sought to answer the focus question: Does the TCSF describe effective teaching in Myanmar? The study included expert review, a teacher survey and case studies of teaching practice in a sample of effective teachers. The report presents key findings and next steps from validation of the TCSF to implementation.

Validation Of The Myanmar Teacher Competency Standards Framework (Tcsf): Phase Iii Case Studies, Anna Dabrowski, Rachel Parker, Amy Berry, Jeaniene Spink

Teacher education

Case studies form the third phase of the validation study for the Myanmar Teacher Competency Standards Framework (TCSF) in advance of its implementation in 2020. As the purpose of the case studies is to ensure that the TCSF is culturally and contextually appropriate, this paper outlines the results of data analysis on the Case Study component. This report focuses on responding to four key research questions: Which standards, minimum requirements and indicators were easiest and hardest to demonstrate? How is effective teaching demonstrated by the selected teachers in Myanmar? How does the evidence of effective teaching differ in different contexts? …

Environmental Scan: Topical Areas To Inform A Future Review Of The Australian Professional Standards For Teachers, Kerry Elliott, Pru Mitchell, Hilary Hollingsworth, Charlotte Waters, Jenny Trevitt

Teaching standards and teacher evaluation

The brief was to conduct a broad sweep of literature that has surfaced since the publication of the Australian Professional Standards for Teachers in 2011, and to consider how this literature could inform any future review of the standards. This multi-pronged scan method identified eleven topics that have increasingly influenced teachers' work over the past decade. The environmental scan report provides a critical review of current educational research and policy literature for each of the 11 topics. Findings provide a description of the topic, key sources, an outline of the changing landscape as it relates to teaching, examples of how …

Teacher Perspectives On The Use Of The Australian Professional Standards For Teachers As Part Of Their Evaluation Process, Damien Barry, Donna Pendergast, Katherine Main

Australian Journal of Teacher Education

Teacher effectiveness has a powerful impact on student performance and a teacher evaluation process that supports professional growth can be a key lever for improving teaching quality. The purpose of this study was to examine teacher perspectives on the use of the Australian Professional Standards for Teachers, when used as part of their evaluation process, and, to determine what other factors may need to be considered in the design and implementation of such a process. A single case study of a school in Victoria, Australia was conducted, using a pre and post interview approach with six teachers. Responses were analysed …

The Van Hiele Analysis Of Curricular Materials: A Comparative Study, Vinay Kumar Kalyankar

Graduate Theses and Dissertations

The goal of this study is to investigate K-8 geometry curriculum standards and textbooks for their effectiveness in preparing students for high school geometry. The basis for the study is van Hiele theory pioneered by the Dutch educators and researchers of geometric understanding in adolescents, Dina van Hiele Geldof and Pierre van Hiele. Another driving force for this dissertation is research into mathematics textbooks and curriculum standards of school mathematics as seen in CCSSM and state specific standards documents, both of which influence the mathematics students have the opportunity to learn.

The K-8 geometry curriculum standards and textbook instructional segments …

Revision Or Re-Vision: Exploring Approaches To The Differentiation Of Qualification Types In The Australian Qualifications Framework, Kate Perkins, Justin Brown, Paul R. Weldon, Louise Wignall

Vocational, adult and professional education research

In March 2019, the Department of Education commissioned the Australian Council for Educational Research (ACER) to conduct a conceptual analysis of the most appropriate way to develop and present a taxonomy of learning outcomes within a qualifications framework; and a technical analysis and revision of the Knowledge, Skills and Application of Knowledge and Skills descriptors used in the Australian Qualifications Framework (AQF). Later this was extended to development of two alternative models that: (a) reflect the new conceptual base developed in Part I; (b) comprise a set of revised domain definitions (along with a new typology for each domain); and …

Options For Reporting Against 4.1.1 When Using National Assessment Programs, Goran Lazendic

Assessment and Reporting

The Sustainable Development Goals are 17 broad aspirations for social and economic development set by the United Nations in 2015. One of these goals, enumerated SDG 4, is to ensure inclusive and equitable quality education and promote lifelong learning opportunities for all. The progress towards each goal is measured by a set of indicators. In monitoring progress towards SDG 4, the UNESCO Institute for Statistics (UIS) is working on an approach to measuring indicator 4.1.1: Proportion of children and young people: (a) in grades 2/3; (b) at the end of primary; and (c) at the end of lower secondary achieving …

Making A Difference:The Promise Of Catholic School Standards, Lorraine A. Ozar, Patricia Weitzel-O’Neill, Teresa Barton, Elizabeth Calteaux, Cristina J. Hunter Phd, Shiya Yi

Journal of Catholic Education

Abstract: The National Standards and Benchmarks for Effective Catholic Elementary and Secondary Schools (NSBECS) were published in 2012 to serve as both a guide and assessment tool for PK-12 Catholic school effectiveness and sustainability; they include research-based general school effectiveness criteria, as well as criteria unique to Catholic school mission and identity. The NSBECS rests on the conviction that adhering to these standards and benchmarks with fidelity will result in highly effective Catholic schools. This Catholic Schools Standards Study Phase 1 began the work of examining the validity of the NSBECS through two national surveys: Survey 1 (2015) focused …

Improving The Rigor Of Online Education: Exploring Characteristics Of Faculty And Syllabi Within An Online Program Assessment Process, Brad J. Hamel

All NMU Master's Theses

As online course enrollments grow, overall perception of rigor still lags compared to that of traditional face-to-face education. The purpose of this research was to tie faculty and syllabi characteristics to the rigor of online courses. This study explored the relationship between faculty and syllabi characteristics and performance on an online entry-level course design quality assurance assessment (pass, pass with concern, or fail). A decision tree analysis was used to predict the relationship of the independent (faculty and syllabi characteristics) and the dependent (entry-level course design assessment) variables. Findings suggest that faculty rank and writing intensive are key characteristics predictive …

Validating Professional Standards For Teachers: A Practical Guide For Research Design: Snapshot Literature Review, Jen Jackson, Yung Nietschke

Dr Jen Jackson

This report aims to establish a strong evidence base for planning a validation study of professional standards for teachers. It presents findings from a rapid "snapshot" review of relevant research literature, to identify previous examples of validation studies, and extract lessons from these about worthwhile methods and considerations in research design. This review was originally conducted to inform the design of a validation study of the draft Myanmar Teacher Competency Standards Framework, but may also have wider relevance to other education systems pursuing similar standards-based reforms. This report presents findings from the literature review. It is divided into four sections: …

Martin School Alumni Survey, Carl Sims

MPA/MPP/MPFM Capstone Projects

The Martin School of Public Policy and Administration is subject to reaccreditation from the Network of Schools of Public Policy, Affairs, and Administration (NASPAA) every seven years. To initiate the reaccreditation process, the Martin School must conduct a programmatic self-study that analyzes the program’s mission, governance, faculty, students, competencies, resources, and communication. The self-study is beneficial not only for the purpose of reaccreditation but also can better inform the school’s strategic planning process

To assist in providing the self-study relevant data, the Martin School can solicit information from its alumni. Alumni data can be used to establish how well the …

Teachers' Perceptions Of School District Professional Development Alignment With National Standards, Arlene Jean Wacha

Walden Dissertations and Doctoral Studies

Despite ongoing delivery of teacher professional development (PD), educational leaders in the local district did not have an objective evaluation whether their implementation of the Marzano causal teacher evaluation model and the iObservation tool training were aligned with accepted national standards for PD effectiveness. Using Vygotsky's constructivist theory as the theoretical framework, the purpose of this quantitative descriptive, correlational study was to describe teachers' perceptions of the alignment of the district's PD with national standards and the relationship between teachers' reported self-efficacy and attitudes toward the PD program. A purposive sample of 80 middle school teachers completed the Learning Forward's …

Assessing Content Mastery: Using Quasi-Experimental Research To Study The Effect That Standards-Based Progress Reporting To Parents May Have On Student Achievement In Science At One Georgia Middle School, Matthew Scott Odell

Doctoral Dissertations and Projects

With the establishment of standards-based testing as the key assessment of student knowledge gains and with the increased accountability first brought by No Child Left Behind (NCLB) and now with the Every Student Succeeds Act (ESSA), the role of the teacher and the methods of assessment have continued to change. To reach the national goals established by NCLB and the state guidelines established to allow the states to guide education in the ESSA, the testing of students’ previous knowledge and the identification of their strengths has become important to show teachers how to meet their students’ needs. More empirical research …

Validating Professional Standards For Teachers: A Practical Guide For Research Design: Snapshot Literature Review, Jen Jackson, Yung Nietschke

Teaching standards and teacher evaluation

This report aims to establish a strong evidence base for planning a validation study of professional standards for teachers. It presents findings from a rapid "snapshot" review of relevant research literature, to identify previous examples of validation studies, and extract lessons from these about worthwhile methods and considerations in research design. This review was originally conducted to inform the design of a validation study of the draft Myanmar Teacher Competency Standards Framework, but may also have wider relevance to other education systems pursuing similar standards-based reforms. This report presents findings from the literature review. It is divided into four sections: …

Developing A Global Health Assessment Collaboration: Ancillary Report, Daniel Edwards, Jacob Pearce, David Wilkinson

Dr Daniel Edwards

This document reports on a project designed to develop an assessment collaboration between medical schools in both Australia and the United Kingdom. The project was funded by the Office for Learning and Teaching (OLT), utilising surplus funding from a broader assessment collaboration project – the Australian Medical Assessment Collaboration (OLT ID12-2482). The Global Health Assessment Collaboration (GHAC) involved five universities in Australia and the United Kingdom (UK). It developed an assessment framework and item specifications, undertook assessment item drafting workshops, built in a process of review and resulted in the development of a focused suite of assessment items. This report …

Assessing Accomplished Teaching With Reliability And Validity: The Acer Portfolio Project, Lawrence Ingvarson

2009 - 2019 ACER Research Conferences

We know that good teachers are worth their weight in gold. But if good teaching is to be truly valued, the teaching profession must be able to demonstrate that it can evaluate itself in ways that are reliable, valid and fair. This capacity is central to any profession. It is also central to lifting the status of teaching, rewarding accomplished teaching and enabling teaching to complete with other professions for our ablest graduates. Recent OECD reports emphasise the necessity of strengthening the teaching profession, which depends upon widespread use of evidence-based teaching practices. Building the capacity for evaluation is the …

Assessing Accomplished Teaching With Reliability And Validity: The Acer Portfolio Project, Lawrence Ingvarson

Dr Lawrence Ingvarson (Consultant)

We know that good teachers are worth their weight in gold. But if good teaching is to be truly valued, the teaching profession must be able to demonstrate that it can evaluate itself in ways that are reliable, valid and fair. This capacity is central to any profession. It is also central to lifting the status of teaching, rewarding accomplished teaching and enabling teaching to complete with other professions for our ablest graduates. Recent OECD reports emphasise the necessity of strengthening the teaching profession, which depends upon widespread use of evidence-based teaching practices. Building the capacity for evaluation is the …

Monitoring And Evaluation: Practitioner Level, Australian Council For Educational Research (Acer)

Education Analytics Service

This Practitioner level module is designed to ensure staff members who manage education investments, can understand the concepts and monitoring and evaluation (M&E) practices needed for designing and supervising education investments. It is recommended that staff complete the Monitoring and Evaluation: Foundation level module as background information to this Practitioner level module. Topics covered in this module include: monitoring versus evaluation; monitoring and evaluation standards; monitoring and evaluation in practice; evaluation considerations; education indicators; monitoring and evaluation reporting; and gender and social inclusion considerations.

Monitoring And Evaluation: Foundation Level, Australian Council For Educational Research (Acer)

Education Analytics Service

The purpose of this module is to provide introductory information about monitoring and evaluation (M&E), including the purpose, application of M&E frameworks, and key issues in education M&E. It provides a foundation to engage in this topic and apply advice from staff with operational or expert levels of knowledge in education M&E.

Principles Of Good Practice In Learning Assessment, Australian Council For Educational Research (Acer), Unesco Institute For Statistics

Monitoring Learning

The Principles of Good Practice in Learning Assessment (GP-LA) are a central element of the international commitment to the management of Sustainable Development Goals (SDGs) 4 data quality for learning outcomes. The GP-LA is an independent articulation of good practices that accommodates the diversity of large-scale learning assessment activities being undertaken throughout the world. Within SDG 4 reporting processes it serves two purposes. First, it serves as the conceptual framework to evaluate the quality of large scale assessments and data from these assessments submitted for SDG 4 reporting. Second, the principles described in the GP-LA will support the diagnosis of …

Implementation Of A Standards-Based Grading Model: A Study Of Parent And Teacher Perceptions Of Success, Amber Dee Wheeler

Graduate Theses and Dissertations

The purpose of this study is to explore the perceptions of parents and teachers regarding the success of a standards-based grading initiative in meeting its goals. Furthermore, findings from this study will be used to inform decisions made in future grade level implementations. Standards-based grading meets all criteria for a problem of practice. The literature situates standards-based grading as a high impact strategy that can affect the entire system. This mixed-methods study will be used to determine practical changes for a school implementing standards-based grading. This study found parents positively responded to the change, and parents understood their child’s current …