Education Commons^™

Pisa Capacity Needs Assessment: Ecuador, Ian Teo

OECD Programme for International Student Assessment (PISA)

This report presents detailed findings of a Capacity Needs Assessment (CNA) for Ecuador, carried out to gain information about Ecuador’s capacity assets and needs to implement the Programme for International Student Assessment (PISA) 2025 successfully. The capacity assets and needs are structured into three dimensions: the enabling environment, the organisational level, and the individual level. Questionnaires for each level are included.

Assessment Moderation: Is It Fit For Purpose?, Lenore Adie

2021-2030 ACER Research Conferences

This presentation outlines the different practices and purposes of assessment moderation and analyses the benefits and issues of these. Two projects are presented; each seeks to improve teacher judgement. Both attempt to redefine what we mean by moderation and how moderation can be conducted to reach diverse and dispersed groups. The first draws on a current Australian Research Council project that is exploring the development of scaled exemplars, the use of commentaries of judgement decisions, and the use of digital platforms to support teachers grading student work. The second draws on the work of the Graduate Teacher Performance Assessment (GTPA) …

Reimagining Classroom Assessment And Feedback To Meet Learner Needs, Fabienne Van Der Kleij

2021-2030 ACER Research Conferences

The power of classroom assessment and feedback to improve student learning outcomes has long been recognised. Yet, decades of research have yielded disappointing and often conflicting outcomes. This presentation challenges traditional conceptions of classroom assessment and feedback as teacher-driven practices. To meet learner needs better, it proposes a student-centred perspective in which students are active and have agency. By drawing on an extensive study of feedback reviews, this presentation illustrates how conceptualisations of feedback have changed over recent decades. This paper provides key insights into how classroom assessment and feedback practices can be changed in ways that are sustainable, afford …

Acer Models Of Writing: Changing The Assessment Mindset, Juliette Mendelovits, Judy Nixon

2021-2030 ACER Research Conferences

This presentation outlines different models of writing assessment that have a proven track record in various programs delivered by ACER. These models include holistic scoring, and partial analytical scoring on a number of criteria. The writing prompts for these models assume extended pieces of writing. The presentation will also introduce a new and alternative model of writing assessment that ACER has implemented in several large-scale assessments. This model is built on shorter pieces of writing from each student, designed to give more accurate and diverse insights into students’ proficiency in writing in a range of text types, within the constraints …

Reimagining The Purpose Of Assessment, Geoff N. Masters

2021-2030 ACER Research Conferences

Assessment is commonly understood as the process of judging how well students have learnt what they have been taught. It comes at the end of a sequence that begins with a curriculum or course syllabus. Teachers are expected to deliver this body of specified content, students are expected to learn it, and assessment is the process of judging and grading students on how well they have learnt what teachers have taught. This is a common view of assessment among students, parents and many teachers. I will argue in this presentation that this traditional understanding of what it means to learn …

Research Conference 2022: Reimagining Assessment: Proceedings And Program, Kylie Burns (Ed.)

2021-2030 ACER Research Conferences

The focus of this year’s Research Conference is on the use of assessment to support improved teaching and learning. The conference is titled ‘reimagining assessment’ because we believe there is a need to transform the essential purposes of educational assessment to provide better information about the deep conceptual learning, skills, competencies, and personal attributes that teachers and schools now have as objectives for student learning and development. Reimagined assessments must now be focused on monitoring learning across this broader range of intended outcomes and provide quality information about the points individuals have reached in their long-term development.

Predicting Language Performance From Narrative Language Samples, Kimberly A. Murphy, Alisha P. Springle, Mollee J. Sultani, Autumn Mcilraith, Language And Reading Research Consortium (Larrc)

Communication Disorders & Special Education Faculty Publications

Purpose: Analysis of narrative language samples is a recommended clinical practice in the assessment of children’s language skills, but we know little about how results from such analyses relate to overall oral language ability across the early school years. We examined the relations between language sample metrics from a short narrative retell, collected in kindergarten, and an oral language factor in grades kindergarten through 3. Our specific questions were to determine the extent to which metrics from narrative language sample analysis are concurrently related to language in kindergarten and predict language through Grade 3. Method: Participants were a sample …

Reflections On The Development Of The Iea Civic And Citizenship Education Studies, Wolfram Schulz

Civics and Citizenship Assessment

This chapter discusses the content, design, and methods of the three IEA studies of civic and citizenship education. It focuses on the changes made between each of these studies and the contexts for this learning area. The IEA Civic Education Study (CIVED) in 1999 was developed soon after the transition to democracy in many participating countries; threats to civil society in the early 2000s such as terrorism influenced the International Civic and Citizenship Education Study (ICCS) in 2009. The design of ICCS in 2016 responded to the increasing importance of social media in civic engagement, the growth in awareness of …

Characterizing Visual Programming Approaches For End-User Developers: A Systematic Review, Mohammad Amin Kuhail, Shahbano Farooq, Rawad Hammad, Mohammed Bahja

All Works

Recently many researches have explored the potential of visual programming in robotics, the Internet of Things (IoT), and education. However, there is a lack of studies that analyze the recent evidence-based visual programming approaches that are applied in several domains. This study presents a systematic review to understand, compare, and reflect on recent visual programming approaches using twelve dimensions: visual programming classification, interaction style, target users, domain, platform, empirical evaluation type, test participants' type, number of test participants, test participants' programming skills, evaluation methods, evaluation measures, and accessibility of visual programming tools. The results show that most of the selected …

Improving Learning: Reconsidering Student Assessment Globally, Amit Kaushik

Assessment and Reporting

Learning assessment plays a critical role in helping to improve learning under different conditions and in varying situations, as an integral part not just of the teaching–learning process, but also of the larger policy environment. While the objective of each type of assessment may differ, its fundamental purpose remains that of helping to determine progress made by each learner in their individual learning path so as to be able to guide and support further development (Masters 2013). Most importantly, utilizing the data emerging from an assessment is critical to its success. The effort of undertaking an assessment is lost when …

Improving Health Science Faculty's Comprehension Of Test Item-Analysis, Tunisia Love

Regis University Student Publications (comprehensive collection)

Background: Differences in testing can lead to student frustration and questionable evaluation of student learning. Inconsistency in testing practices include differing levels of Bloom’s taxonomy, Faculty-written items versus publisher test bank items, and variations in grading. Failure to provide quality student learning assessment and evaluation tools and methods can negatively impact both student and program outcomes. Performing analysis of test items can improve quality of and consistency in evaluation of student learning.

Objective/Purpose of the Project: This quality improvement DNP project investigated the impact of an educational intervention on multiple choice question (MCQ) test-item analysis on the Health …

Learning Through Play Experience Tool: Contextual Data Tools, Australian Council For Educational Research (Acer)

Monitoring Learning

The Learning through Play (LtP) Experience Tool was developed for use at LEGO House to observe and measure the quality of children’s play experiences. The LtP Experience Tool includes a set of Contextual Data Tools. The primary purpose of the Contextual Data Tools: "is to put the LtP Experience Tool into ‘context’, by emphasising that we are not assessing the individual child assuming that there is something wrong with the child’s performance or behaviour. But that we are measuring the child to understand whether anything should be changed in the context/environment/facilitation around the child". The Contextual Data Tools …

The Science Behind The Art Of Teaching: Evaluation As Inspiration, Michele Bruniges Am

2009 - 2019 ACER Research Conferences

Teachers across Australia inspire students to love learning. Our best teachers are constantly evaluating their impact on learning outcomes and adapting their practice – balancing the art and science of teaching. As we move rapidly towards the third decade of the 21st century, there is more pressure than ever for all teachers to deliver both deep discipline knowledge and the skills students need to survive and thrive in the workplace of the future. We need to use technology and data to support teachers to maximise learning outcomes for their students. This has to be done in a way that helps …

Measures Of Characteristics And Skills Associated With Learning Through Play: Findings From The Literature, Rachel Parker, Lisa Van Beeck, Janine Callanan

Monitoring Learning

This literature review was commissioned by the LEGO Foundation to inform the revision of the Learning through Play (LtP) Experience Tool, which was developed for use at LEGO House to observe and measure the quality of children’s play experiences. This review aims to determine the extent to which the LEGO Foundation’s tool reflects good practice regarding instrument design and administration, and whether tools that measure similar characteristics using similar methods already exist

Developing A Global Health Assessment Collaboration: Ancillary Report, Daniel Edwards, Jacob Pearce, David Wilkinson

Dr Daniel Edwards

This document reports on a project designed to develop an assessment collaboration between medical schools in both Australia and the United Kingdom. The project was funded by the Office for Learning and Teaching (OLT), utilising surplus funding from a broader assessment collaboration project – the Australian Medical Assessment Collaboration (OLT ID12-2482). The Global Health Assessment Collaboration (GHAC) involved five universities in Australia and the United Kingdom (UK). It developed an assessment framework and item specifications, undertook assessment item drafting workshops, built in a process of review and resulted in the development of a focused suite of assessment items. This report …

Equipping Teachers With Tools To Assess And Teach General Capabilities, Claire Scoular

2009 - 2019 ACER Research Conferences

There is wide recognition that students need to be equipped with appropriate social and cognitive skills demanded by society and the workforce. The unresolved question is how to do this. Many education systems globally are addressing this demand by including skills such as critical thinking, problem-solving, collaboration, and creativity into curriculum documents or supplementary materials. However, there is little research to guide educators in teaching such skills at school level. The need to develop practical solutions for assessing and teaching social and cognitive skills, broadly classified under the umbrella ‘21st-century skills’ or ‘general capabilities’, is ever increasing. An integrated approach …

Equipping Teachers With Tools To Assess And Teach General Capabilities, Claire Scoular

Dr Claire Scoular

There is wide recognition that students need to be equipped with appropriate social and cognitive skills demanded by society and the workforce. The unresolved question is how to do this. Many education systems globally are addressing this demand by including skills such as critical thinking, problem-solving, collaboration, and creativity into curriculum documents or supplementary materials. However, there is little research to guide educators in teaching such skills at school level. The need to develop practical solutions for assessing and teaching social and cognitive skills, broadly classified under the umbrella ‘21st-century skills’ or ‘general capabilities’, is ever increasing. An integrated approach …

Monitoring Learning Against The Sustainable Development Goals, Australian Council For Educational Research (Acer)

Monitoring Learning

The Global Alliance to Monitor Learning (GAML) is an international collaboration to improve learning around the world, led by the UNESCO Institute for Statistics (UIS). The Australian Council for Educational Research Centre for Global Education Monitoring (ACER-GEM) has a major role in supporting GAML, as the UIS’s technical partner. This publication outlines ACER’s role in the work plan of the Global Alliance to Monitor Learning.

System-Wide Analysis Of Assessment Practices (Swaap): Concept Note, Australian Council For Educational Research

Global education monitoring

This note is concerned with the development of a process to analyse how assessment practices are integrated into education systems in order for assessment outcomes to be useful for evidence based decision making throughout the system. It examines the quality of learning assessment and the extent to which it is embedded into an education system and acts as an effective contributor to the improvement of learning outcomes. The process is referred to as the System-Wide Analysis of Assessment Practices (SWAAP). The primary objective of SWAAP is to support education systems to develop national assessment strategies that generate meaningful data and …

Developing A Professional Certification System For School Principals: The Principals Australia Institute Certification Project, Lawrence Ingvarson

Dr Lawrence Ingvarson (Consultant)

In 2014, the Principals Australia Institute (PAI) decided to develop a national system for providing professional certification to accomplished school principals, based on the AITSL Australian Professional Standard for Principals (the Standard). ACER has been assisting the PAI in the development of valid and reliable methods whereby principals can demonstrate how they meet the Standard. This work has included conceptualising the system, developing an assessment and evaluation framework for certification, and developing guidelines for three portfolio initiatives linked to the Australian Professional Standard for Principals. The portfolio initiatives were field-tested in 2015 and a group of principals was trained to …

Developing A Professional Certification System For School Principals: The Principals Australia Institute Certification Project, Lawrence Ingvarson

2009 - 2019 ACER Research Conferences

In 2014, the Principals Australia Institute (PAI) decided to develop a national system for providing professional certification to accomplished school principals, based on the AITSL Australian Professional Standard for Principals (the Standard). ACER has been assisting the PAI in the development of valid and reliable methods whereby principals can demonstrate how they meet the Standard. This work has included conceptualising the system, developing an assessment and evaluation framework for certification, and developing guidelines for three portfolio initiatives linked to the Australian Professional Standard for Principals. The portfolio initiatives were field-tested in 2015 and a group of principals was trained to …

Developing A Global Health Assessment Collaboration: Ancillary Report, Daniel Edwards, Jacob Pearce, David Wilkinson

Higher education research

This document reports on a project designed to develop an assessment collaboration between medical schools in both Australia and the United Kingdom. The project was funded by the Office for Learning and Teaching (OLT), utilising surplus funding from a broader assessment collaboration project – the Australian Medical Assessment Collaboration (OLT ID12-2482). The Global Health Assessment Collaboration (GHAC) involved five universities in Australia and the United Kingdom (UK). It developed an assessment framework and item specifications, undertook assessment item drafting workshops, built in a process of review and resulted in the development of a focused suite of assessment items. This report …

Session J: Assessment In Interactive Learning Environments, Michael Timms, Jason Lodge

Dr Michael J Timms

There is an increasing interest in using digital technologies to create interactive learning environments (ILEs) that both teach and assess student skills that are hard or impossible to assess using ‘static’ items such as traditional, multiple-choice questions. These interactive learning environments try to do two things simultaneously: firstly, to monitor the learning of the student in real time, providing feedback to help the student progress through the learning task; and secondly, to use the information gathered during the learning to make judgements about where the student is in learning of the topic. Essentially, ILEs draw upon the same source of …

Assessment In Interactive Learning Environments, Michael Timms, Jason Lodge

2009 - 2019 ACER Research Conferences

There is an increasing interest in using digital technologies to create interactive learning environments (ILEs) that both teach and assess student skills that are hard or impossible to assess using ‘static’ items such as traditional, multiple-choice questions. These interactive learning environments try to do two things simultaneously: firstly, to monitor the learning of the student in real time, providing feedback to help the student progress through the learning task; and secondly, to use the information gathered during the learning to make judgements about where the student is in learning of the topic. Essentially, ILEs draw upon the same source of …

A Review Of International Large-Scale Assessments In Education: Assessing Component Skills And Collecting Contextual Data, John Cresswell, Ursula Schwantner, Charlotte Waters

OECD Programme for International Student Assessment (PISA)

The OECD has initiated PISA for Development (PISA-D) in response to the rising need of developing countries to collect data about their education systems and the capacity of their student bodies. This report aims to compare and contrast approaches regarding the instruments that are used to collect data on (a) component skills and cognitive instruments, (b) contextual frameworks, and (c) the implementation of the different international assessments, as well as approaches to include children who are not at school, and the ways in which data are used. It then seeks to identify assessment practices in these three areas that will …

Listening For The Squeaky Wheel: Designing Distance Writing Program Assessment, Virginia M. Tucker

English Faculty Publications

Distance writing programs still struggle with assessment strategies that can evaluate student writing as well as their ability to communicate about that writing with peers at a distance. This article uses Kim, Smith and Maeng's 2008 distance education program assessment scheme to evaluate a single distance writing program at Old Dominion University. The program's specific assessment needs include the ability to determine how well students are developing expert insider prose and working together as a virtual community. Kim, Smith and Maeng's assessment scheme was applied to six courses within the writing program, revealing that programmatic assessment weaknesses included providing varied …

An Analysis Of Evaluation Plans In A Federally Funded Leadership Preparation Program: Implications For The Improvement Of Practice, Karen Sanzo

Educational Leadership & Workforce Development Faculty Publications

This study is a content analysis of selected federally funded leadership preparation program evaluation proposals for the 2008, 2009, and 2010 School Leadership Program grants. The United States Department of Education (USDE) began awarding funding in 2002 to develop and implement preparation programs for aspiring and current assistant principals and principals. These innovative leadership preparation programs are expected to influence school leaders in a way that enables them to positively impact student achievement and the schools they serve. In order to determine effectiveness, each grant proposal puts forth a comprehensive program evaluation designed to measure the grant's adherence to the …

Assessing The Evaluation Of Library Instruction Using A Business Model, Barbara Petersohn

Georgia International Conference on Information Literacy

The business sector has traditionally evaluated their training and instruction programs on a model that considers 4 levels of success (Kirkpatrick). These levels can be described as success based on 1) appeal, 2) learning outcomes 3) integration of learning into daily work routines, or 4) overall impact on / improvement of the organization. This paper describes how this rubric can be generally, and ideally, translated and applied to instruction programs in libraries to determine what levels of evaluation are attempted by libraries. And, based on a review and analysis of case studies of the literature of library instruction, considers questions …

Indigenous Languages Programmes In Australian Schools - A Way Forward, Nola Purdie, Tracey Frigo, Clare Ozolins, Geoff Noblett, Nick Thieberger, Janet Sharp

Indigenous Education Research

Currently, over 16,000 Indigenous students and 13,000 non-Indigenous students located in 260 Australian schools are involved in an Indigenous language program. More than 80 different Indigenous languages are taught. This project sought to present practice which would strengthen the quality of Indigenous language programs in schools. The report consists of a literature review, a mapping exercise to document current practices relating to Indigenous languages in Australian schools, an analysis of existing models of teacher preparation, and six case studies of good practice examples.

International Partnership In Educational Strategic Planning And Evaluation: The Muffles College Project, Brian Satterlee

Brian Satterlee

The Florida Association of voluntary Agencies for Caribbean Action was formed by the state’s Governor to increase cooperation with Caribbean nations and provide on-site technical assistance and training in health, agriculture, social services, and education. In 1996, the Association conducted a project with Muffles College, a junior college in Belize, to develop a strategic planning workshop for the college’s faculty, administration, and constituents and to develop an implement an institutional evaluation process for the college. The project resulted in the following outcomes: (1) the strategic planning workshop helped staff identify the college’s internal strengths, such as the discipline of the …