Educational Assessment, Evaluation, and Research Commons^™

Literature Review Relating To The Current Context And Discourse On Indigenous Cultural Awareness In The Teaching Space: Critical Pedagogies And Improving Indigenous Learning Outcomes Through Cultural Responsiveness, Jacynta Krakouer

Indigenous Education Research

This literature review seeks to provide an overview of current understandings and discourse about culturally responsive teaching and cultural awareness in education in Australia. Although some of the literature considered within this review is from an international perspective, the purpose of this review is to provide background information regarding culturally responsive teaching with Aboriginal and Torres Strait Islander students. This literature review was completed in a short time frame and is by no means exhaustive on the topic of culturally responsive teaching and cultural awareness with Indigenous students. It was completed as part of a contract with the VET Development …

National Assessment Program : Ict Literacy Years 6 & 10 Report 2014, Julian Fraillon, Wolfram Schulz, Eveline Gebhardt, John Ainley

ICT - Digital Literacy

Literacy in information and communication technology (ICT) is critical to students as they progress through schooling and enter a world in which information technology will be ubiquitous. Work, health care, family finances, learning and social interaction will all depend on competence in ICT. To assess progress in this crucial part of student learning, ACARA conducts a National Assessment Program (NAP) aimed at measuring ICT literacy. Every three years since 2005, a sample of Year 6 and Year 10 students from across Australia have been tested to determine their ICT knowledge, understanding and skills and their ability to use ICT creatively, …

Completing University In A Growing Sector: Is Equity An Issue?, Daniel Edwards, Julie Mcmillan

Higher education research

Access to university has always been an issue for students from disadvantaged backgrounds. In the recent context of an expanding higher education system in Australia, some accessibility issues have been alleviated. This context offers an opportunity to explore the pathways of disadvantaged students through university. In this expanded system, will disadvantaged students be more or less likely to complete university? Will demographic or enrolment characteristics influence the likelihood of these students to complete? This report details the findings from a research project funded by the National Centre for Student Equity in Higher Education (NCSEHE) that explores new data tracking student …

Hacking Assessment, Phillip Dawson

2009 - 2019 ACER Research Conferences

Hackers exploit weaknesses in a system to achieve their own goals. In this paper I argue that hacking presents a significant threat to the growing world of online assessment. This threat needs to be addressed through a variety of means; technological anti-hacking approaches will not be sufficient. The most effective ways to prevent hacking may be changes to the assessment tasks themselves to make hacking less tempting; these approaches also have a range of positive side effects in terms of authenticity, transparency of criteria, and ensuring tasks involve work beyond the exam. I conclude with a brief exploration of the …

Translating Rich Learning Assessments Into Certified Results And University Selection Devices, Gabrielle Matters

2009 - 2019 ACER Research Conferences

There are challenges in designing a set of high-quality processes in senior assessment and tertiary entrance that meet the needs of future senior secondary school students and future users of the certified results of learning assessments. Assessment and selection arrangements should look to the future rather than backwards to arrangements that might have existed in the past or that presently operate, unexamined, in other places. Teachers need to be convinced that the richness of students’ learning assessments will not be lost or transmogrified in any new processes for grading or ranking. A set of principles should guide the design of …

Assessing General Capabilities, Julian Fraillon, Juliette Mendelovits

2009 - 2019 ACER Research Conferences

There is growing interest in general capabilities and cross-curricular learning outcomes such as literacy in information and communication technologies, creative thinking and collaborative and individual problem-solving. As the expectation for such competencies to be taught in schools has increased, so has the need for teachers and schools to validly and reliably assess student learning in those areas, and to report on them in ways that teaching and learning. In this presentation examine the challenges of assessing on student learning and learning growth capabilities and cross-curricular learning present approaches used in research some of these challenges and reflect can be applied …

Stealth Assessment In Video Games, Val Shute

2009 - 2019 ACER Research Conferences

Games can be powerful vehicles to support learning, but their success in education hinges on getting the assessment part right. In this presentation, I will explore how games can use stealth assessment to measure and support the learning of competencies critical for the future. I will discuss what stealth assessment is, why it is important, and how to develop and accomplish it. I will also provide examples within the context of a game called Physics Playground that I designed and developed with my team. I’ll share what has been learned by recent research on stealth assessments in games, including: Does …

Assessment In Interactive Learning Environments, Michael Timms, Jason Lodge

2009 - 2019 ACER Research Conferences

There is an increasing interest in using digital technologies to create interactive learning environments (ILEs) that both teach and assess student skills that are hard or impossible to assess using ‘static’ items such as traditional, multiple-choice questions. These interactive learning environments try to do two things simultaneously: firstly, to monitor the learning of the student in real time, providing feedback to help the student progress through the learning task; and secondly, to use the information gathered during the learning to make judgements about where the student is in learning of the topic. Essentially, ILEs draw upon the same source of …

Measuring What Matters: Challenges And Opportunities In Assessing Science Proficiency, James W. Pellegrino

2009 - 2019 ACER Research Conferences

A key challenge in shaping science learning for the future will be to develop new measures of learning that take into account what it means to be proficient in science (Pellegrino, 2013). The emergent view on proficiency, grounded in learning sciences research, emphasises using and applying knowledge in the context of disciplinary practice. Referred to as knowledge-in-use, this perspective on science proficiency is a centrepiece of the United States’ National Research Council’s (NRC) Framework for K–12 Science Education (NRC, 2012), embodied in the new US national standards (NGSS Lead States, 2013) and emphasised in the recently released NRC report on …

Assessment Standards, ‘Intentional Alignment’, And Dialogic Inquiry, Claire Wyatt-Smith

2009 - 2019 ACER Research Conferences

Internationally, the policy move towards standards-aligned instruction is gaining momentum. In Australia, standards have assumed unprecedented prominence in education policy relating both to classroom practice and to teacher preparation and career progression. The move is also evident in the United States, where the lure of standards to inform improvement is clear: significant investment has been committed to longitudinal research to examine at state and district levels the desirable conditions for implementing standards, their impact on developing college- and career-ready teachers, and in turn, the impact on teacher instruction and student outcomes. Moves such as this are occurring in the absence …

Should Generic Curriculum Capabilities Be Assessed?, Rosemary Hipkins

2009 - 2019 ACER Research Conferences

Both Australia and New Zealand have recently taken up the idea of ‘key competencies’ (‘capabilities’ in the Australian national curriculum) initially proposed by the Organisation for Economic Co-operation and Development. In both countries we have made them our own by adapting them to suit our own educational contexts. People often say that these capabilities won’t be taken seriously unless they are assessed. So whether, and how, to assess them continue to be vexed questions. In this paper I argue that capabilities are more appropriately seen as changing the curriculum rather than adding to it. If we are serious about preparing …

Learning Assessments: Designing The Future, Geoff N. Masters

2009 - 2019 ACER Research Conferences

Processes for assessing student learning are undergoing fundamental transformation.This presentation will consider three developments which can be expected to shape how student learning is assessed in the future. First is fundamental change in how assessment is conceptualised and approached, with a focus on monitoring learning. Second is growing interest in the assessment of a broader range of skills and attributes than those addressed in most current assessment efforts. Third is advances in technology which are opening the door to new ways of gathering information about student learning, including through records of real-time interactions in online learning environments. In ACER’s Centre …

Creating Stealth Assessments, Val Shute, Michael Timms

2009 - 2019 ACER Research Conferences

In this workshop, participants will learn how to create stealth assessments to measure student performance during interactions within computer-based learning environments, like digital games or intelligent tutoring systems. These measures are then used to estimate various competencies, including hard-to-measure constructs like creativity, persistence, problem-solving and systems thinking. First, we will explain how evidence-centred design can be used as a theoretical approach to designing such assessments. Next, we’ll illustrate how evidence-centred design was applied in the development of stealth assessment within a particular game (using the example of Plants vs. Zombies 2). Participants will have a chance to create their …

Improving Assessment For Indigenous Students, Alison Quin

2009 - 2019 ACER Research Conferences

This workshop will introduce a teaching scenario and existing unit of work built around Indigenous culturally inclusive practices. Participants will work in small groups to devise assessment items in a scaffolded process. The development of assessment tasks will be informed by clarifying Indigenous education, the process of building relationships between schools and Indigenous communities that contribute to co-developed units of work, and principles of Indigenous culturally inclusive practice such as place-based, community-grounded and group production practices.

Assessing Young Children's Literacy And Mathematics Understandings, Collette Taylor, Joanne Mulligan, Maurice Walker, Prue Anderson, Marion Meiers

2009 - 2019 ACER Research Conferences

Current research indicates that young children are capable of developing mathematical concepts and reasoning much earlier than previously considered. Moderated by Professor Collette Tayler, this symposium on assessing young children's literacy and mathematics understandings is in three parts. Firstly the authors explore the Pattern and Structure Assessment (PASA). PASA is an early mathematical assessment interview which focuses on a range of concepts and processes and is linked with mathematical attainment in the ACER Progressive Achievement Tests in Maths (PATMaths). Secondly, the authors report on the piloting of an early-years technology-based tool called the Digital Early Reading and Mathematics Assessment (DERMA). …

New Measures For An Old Friend: A Learning Progression For Ict Literacy, Mark Wilson

2009 - 2019 ACER Research Conferences

This workshop will present new thinking and new results from the work of the Assessment and Teaching of 21st-Century Skills (ATC21S) project, on the learning progression for information and communications technology literacy — learning in digital networks. This project, initially sponsored by Cisco, Intel and Microsoft, aimed to help educators around the world enable students with the skills to succeed in future career and college goals. The workshop will be structured to show the how the development of the new ideas and measures for ICT literacy followed the logic of the assessment system developed by the Berkeley Evaluation and Assessment …

Acer Research Conference Proceedings (2015), Acer

2009 - 2019 ACER Research Conferences

It is indeed timely that Research Conference 2015 addresses the theme Learning Assessments: Designing the future. It is six years since our Research Conference considered issues in assessment, and the landscape is being significantly transformed. Not only is Australia’s school curriculum changing, but related issues of teaching quality and assessment practice are hot topics here and in many other countries. This transforming landscape includes changes in thinking about the fundamental purposes of assessment; growing demands for the assessment of a broader range of student skills and capabilities; and new technologies that allow us to gather and visualise information about student …

‘Big Five’ Challenges In School Education, Geoff N. Masters Ao

Teacher columnist – Geoff Masters

Real reform and significant progress in improving the quality and equity of Australian schooling depend on tackling our deepest and most stubborn educational challenges, writes Professor Geoff Masters AO.

International Computer And Information Literacy Study: Icils 2013: Technical Report, Julian Fraillon, Wolfram Schulz, Tim Friedman, John Ainley, Eveline Gebhardt

ICT - Digital Literacy

The International Computer and Information Literacy Study (ICILS) studied the extent to which young people have developed computer and information literacy (CIL) to support their capacity to participate in the digital age. ICILS systematically investigated differences in CIL outcomes across the participating countries and looked at how these countries provide CIL-related education. The study also explored differences within and across countries with respect to the relationship between the outcomes of CIL education and student characteristics and school contexts. Chapter 1 of this report presents an overview of the International Computer and Information Literacy Study. Chapters 2 and 3 are concerned …

Imperatives In Schools Funding: Equity, Sustainability And Achievement, Lyndsay Connors, Jim Mcmorrow

Australian Education Review

AER 60 takes the 2011 Gonski Review of Funding for Schooling as a vantage point, from which to consider the national funding of schools in Australia, past and future.

Section 1 outlines the educational values and perspectives that have underpinned school funding policies and programs and describes major themes in the evolution of schools funding – quality, equity and choice – providing a general guide for subsequent analyses. Section 2 provides a policy history of schools funding 1964 – 2011, during which time the Commonwealth government emerged as a significant funding partner. It analyses the political forces and recurring themes, …

Australian Students In A Digital World, Sue Thomson

Policy Insights

This century has seen continued exponential growth in the use of digital technologies. In Australia, the proportion of students having access to a computer at home rose from about 91 per cent in 2000 to over 99 per cent in 2013, and access to the internet grew from 67 per cent in 2000 to 98 per cent in 2013. According to the 2013 report on the International Association for the Evaluation of Education Achievement’s (IEA) International Computer and Information Literacy Study (ICILS), Australia had the highest percentage of students who used computers at school at least once a week (81%), …

Work Integrated Learning In Stem In Australian Universities: Final Report: Submitted To The Office Of The Chief Scientist, Daniel Edwards, Kate Perkins, Jacob Pearce, Jennifer Hong

Higher education research

The Australian Council for Educational Research (ACER) undertook this study for the Office of the Chief Scientist (OCS). It explores the practice and application of Work Integrated Learning (WIL) in STEM, with a particular focus on natural and physical sciences, information technology, and agriculture departments in Australian universities. The project involved a detailed ‘stocktake’ of WIL in practice in these disciplines, with collection of information by interview, survey instruments, consultation with stakeholders and literature reviews. Every university in Australia was visited as part of this project, with interviews and consultation sessions gathering insight from more than 120 academics and support …

Evaluation Of Ict In Education In Papua Province (Acdp-045): Final Report, Susanne Owen, John Hunt, Eda Simanjuntak, Eko Sediyono, Prapto Yuwono, Sri Yulianto, Wigati Yektinintys Modouw

ACER Indonesia

The Evaluation of Information and Communication Technologies (ICT) in Education in Papua Province study is a large-scale research project for the Republic of Indonesia’s Education Sector Analytical and Capacity Development Partnership (ACDP). The key objectives of the Study are: to measure the effectiveness of the TV-Edukasi (Education) (TV-E) and other ICT programs in the Province and to determine their impact on the learning outcomes of school children and adult learners; and to make recommendations to plan for further ICT in education initiatives.

Challenging Our Most Able Students, Geoff N. Masters Ao

Teacher columnist – Geoff Masters

Professor Geoff Masters AO discusses identifying and addressing the needs of individual learners in Australian schools.

Assessments To Support Quality Teaching And Learning, Charlotte Waters

International Developments

The focus of post-2015 development goals on educational quality for all is directing attention to assessments that support quality teaching and learning, as Charlotte Waters explains.

The Cure For Early Grades Assessment Difficulties? Take A Tablet, Maurice Walker

International Developments

Monitoring educational development in the early years of schooling is vital if practitioners, and policy makers, are to support students’ learning, but the assessment of student achievement in developing countries can be a logistical headache. Maurice Walker reports on an innovative approach to assessment using tablets that is addressing that.

International Developments (No.5) 2015

International Developments

This issue of International Developments focuses on the work of the Centre for Global Education Monitoring (GEM) at ACER. GEM is tracking progress in the provision and quality of schooling through the systematic and strategic collection of data on educational outcomes, and factors that influence these. Table of contents for this issue: (a) Data to inform policy; (b) The cure for early grades assessment difficulties? Take a tablet; (c) Measuring growth in a world of universal education; (d) Assessments to support quality teaching and learning.

Enhancing Cross-Border Higher Education Institution Mobility In The Apec Region, Sarah Richardson

Higher education research

This report presents findings of a study of the policy context around higher education institution (HEI) mobility in eight APEC economies – China, Indonesia, Malaysia, Mexico, Peru, the Philippines, Singapore and Viet Nam. The report provides insights into different policy contexts gathered from desktop research and from interviews with 68 participants. These all had direct involvement in HEI mobility in the eight economies in the study, including through senior positions in ministries of education, in bodies aligned to ministries, in accreditation and quality assurance agencies, and in foreign and local HEIs. The report identifies regulations, practices, policies, legislative contexts and …

Supporting Teacher Development : Literature Review, Kate Reid, Elizabeth Kleinhenz

Initial Teacher education

This literature review identifies and evaluates evidence on the effectiveness of approaches to improving teacher quality in developing countries. It examines four categories of assistance for improving teacher quality: teacher development policies, pre-service education and training, in-service professional development, and school-based support. As far as possible, the review highlights practices that demonstrate effectiveness.

Learning Assessments : Designing The Future, Geoff N. Masters Ao

Teacher columnist – Geoff Masters

Professor Geoff Masters discusses recent developments in assessing student learning and how they will shape the future of assessment.