Educational Assessment, Evaluation, and Research Commons^™

Assessing Reading: How Assessment Can Be Used To Target Teaching And Enhance Understanding Of Reading Comprehension, Sandra Knowles

2021-2030 ACER Research Conferences

The skills demonstrated by a proficient reader are not easy to untangle. Current research acknowledges that reading comprehension is a highly complex area of ability, one that needs to be understood as the coordination of several integrated processes. Using example test questions and data, this presentation explores how assessment can help us make sense of reading comprehension in a way that curricula and commonly used teaching strategies cannot. Assessment is evidence that informs us about the skills involved in the reading process, how they relate to each other, and how they develop in complexity. When assessment is understood in this …

Evidence-Informed Early Language And Literacy Practice: Implementation Of A New Formative Assessment For Early Language And Literacy Development, Sarah Groom, Mary-Ruth Mendel, Eric Brace, Philippa Pryor

2021-2030 ACER Research Conferences

With one in five Australian children starting school developmentally vulnerable, there is a need in this critical period for visibility over children’s oral language and early literacy development. The Australian Literacy and Numeracy Foundation (ALNF) partnered with a Northern Territory primary school to implement the Early Language and Literacy Development Index (ELLDI), a newly-created valid and reliable assessment which places children on a progression scale for oral language and early literacy development and provides meaningful, accessible recommendations for educational practice. Twenty-three children in the school’s transition cohort were measured using the ELLDI at the beginning and end of the school …

Making Learning Visible: Moving From Nouns To Verbs, Diane Debacker

2021-2030 ACER Research Conferences

This presentation focuses on the need to make learning visible for all constituents – learners, educators, employers, policymakers and future employees – with a focus on the work of the Center for Certification and Competency-Based Education (C3Be) at the University of Kansas (USA). As K–12 and higher education move to a post-pandemic world, it’s imperative we find different ways to identify learner outcomes and evaluate learning. By moving from nouns to verbs; by moving from course titles to competencies; by moving from transcripts to learner outcomes, learning becomes visible. International educator and founding director of C3Be, Dr Diane DeBacker shows …

Assessment Moderation: Is It Fit For Purpose?, Lenore Adie

2021-2030 ACER Research Conferences

This presentation outlines the different practices and purposes of assessment moderation and analyses the benefits and issues of these. Two projects are presented; each seeks to improve teacher judgement. Both attempt to redefine what we mean by moderation and how moderation can be conducted to reach diverse and dispersed groups. The first draws on a current Australian Research Council project that is exploring the development of scaled exemplars, the use of commentaries of judgement decisions, and the use of digital platforms to support teachers grading student work. The second draws on the work of the Graduate Teacher Performance Assessment (GTPA) …

The Transformational Change Of Adaptive Assessment, Jarrod Hingston, Toby Newton

2021-2030 ACER Research Conferences

Organisational change theory often discusses the differences between adaptive change and transformational change; adaptive change being the process of incremental shifts that are intended to drive improvement, while transformational change requires major alterations in processes. In 2019, ACER embarked on a mission to develop a new adaptive testing and reporting system, with the goal of delivering an assessment that would improve the precision of results for every student. While the intention was to introduce an adaptive change in the assessments, the project truly transformed the way that ACER delivers assessments and reports results, a result recognised by the project being …

Sharing And Securing Learners' Performance Standards Across Schools, Richard Kimbell

2021-2030 ACER Research Conferences

Assessing learners’ performance makes very different demands upon teachers depending on the purpose and the context of the assessment. But common to all assessment is some sense of what ‘quality’ looks like. Most often teachers engage in formative assessments in the classroom, and the familiar standards of the classroom are adequate for this purpose. However if teachers are to undertake external, nationally regulated assessment then some sense of a national standard of quality is required. But there are very limited mechanisms by which teachers can acquire this understanding, so they use their best judgement, and standards vary from school to …

An Innovative Method For Teachers To Formatively Assess Writing Online, Dr Sandy Heldsinger, Stephen Humphry

2021-2030 ACER Research Conferences

Assessment is an integral component of effective teaching and a teacher’s professional judgement influences all routine aspects of their work. In the last 20 years, there has been considerable work internationally to support teachers in using assessment to improve student learning. However, there is a pressing issue that impedes teacher professional judgement being exploited to its full potential. The issue relates to teacher assessments in the context of extended performances such as essays and arises from the complexity of obtaining reliable or consistent teacher assessments of students’ work. Literature published in the United States, England and Australia details evidence of …

Using Pat Data To Inform Teaching And Learning, Marc Kralj, Rachel Felgate, Shani Sniedze, Caithlin Power, Grant Barclay, Darren Leech

2021-2030 ACER Research Conferences

ACER works with thousands of Australian schools to set up Progressive Achievement Test (PAT) functionality, assign and administer tests, and to assist teachers to understand and interrogate the resulting test data and act upon findings. This session provides participants with an opportunity to hear about a school’s journey, and discuss this with the ACER team and our PAT school guest. ACER researchers have evaluated and documented assessment implementation and professional development opportunities, observing this school’s ability to track student and educator progress over time, which is a critical element in ACER’s Progressive Achievement approach. ACER’s educational consultants will demonstrate how …

Reimagining Classroom Assessment And Feedback To Meet Learner Needs, Fabienne Van Der Kleij

2021-2030 ACER Research Conferences

The power of classroom assessment and feedback to improve student learning outcomes has long been recognised. Yet, decades of research have yielded disappointing and often conflicting outcomes. This presentation challenges traditional conceptions of classroom assessment and feedback as teacher-driven practices. To meet learner needs better, it proposes a student-centred perspective in which students are active and have agency. By drawing on an extensive study of feedback reviews, this presentation illustrates how conceptualisations of feedback have changed over recent decades. This paper provides key insights into how classroom assessment and feedback practices can be changed in ways that are sustainable, afford …

Acer Models Of Writing: Changing The Assessment Mindset, Juliette Mendelovits, Judy Nixon

2021-2030 ACER Research Conferences

This presentation outlines different models of writing assessment that have a proven track record in various programs delivered by ACER. These models include holistic scoring, and partial analytical scoring on a number of criteria. The writing prompts for these models assume extended pieces of writing. The presentation will also introduce a new and alternative model of writing assessment that ACER has implemented in several large-scale assessments. This model is built on shorter pieces of writing from each student, designed to give more accurate and diverse insights into students’ proficiency in writing in a range of text types, within the constraints …

Reimagining The Purpose Of Assessment, Geoff N. Masters

2021-2030 ACER Research Conferences

Assessment is commonly understood as the process of judging how well students have learnt what they have been taught. It comes at the end of a sequence that begins with a curriculum or course syllabus. Teachers are expected to deliver this body of specified content, students are expected to learn it, and assessment is the process of judging and grading students on how well they have learnt what teachers have taught. This is a common view of assessment among students, parents and many teachers. I will argue in this presentation that this traditional understanding of what it means to learn …

Research Conference 2022: Reimagining Assessment: Proceedings And Program, Kylie Burns (Ed.)

2021-2030 ACER Research Conferences

The focus of this year’s Research Conference is on the use of assessment to support improved teaching and learning. The conference is titled ‘reimagining assessment’ because we believe there is a need to transform the essential purposes of educational assessment to provide better information about the deep conceptual learning, skills, competencies, and personal attributes that teachers and schools now have as objectives for student learning and development. Reimagined assessments must now be focused on monitoring learning across this broader range of intended outcomes and provide quality information about the points individuals have reached in their long-term development.

Relationships Between And Student Perceptions Of Self-Efficacy, Growth Mindset, And Mathematics Identity Of Adolescents In A Rural South Georgia Charter School, Carrie Cooper Pierce

Theses and Dissertations

This mixed methods study investigated the relationships between the constructs of self-efficacy, growth mindset, and mathematics identity among middle school students at a South Georgia charter school. Constructivist grounded theory was used to investigate the strength of these constructs along with student perceptions of what factors make one a math person. The study was an explanatory sequential design consisting of an online survey followed by student interviews. Ninety-one students participated in the survey. Multiple linear regression was used to determine whether a correlation existed between the variables as well as among student subgroups. Self-efficacy was found to be a predictor …

Could The Inclusion Of Certain Building Information And Modelling Aspects Into The Leaving Certificate Engineering Syllabus, Aid The Transition Of Students Into Third Level Education, When Choosing A Course Within The Construction Industry, Joseph Boyle, Deborah M. Brennan

Capstone Reports

The paper investigates the effects of implementing several Building Information and Modelling (BIM) aspects into the Leaving Certificate Engineering Syllabus and examines the impact this has on students choosing third level Architectural Engineering and Construction (AEC) industry courses with the desire of improving the shortage in the sector. The method and content of the study was based on academic research on second level education in other European countries through the lens of a Literature Review. A sixteen-week pilot program was trialed with over one hundred students at Senior Cycle. The outcomes of the study were critically evaluated through stakeholder interviews …

Effect On Student Achievement For Middle School Math Students Using Blackboard, Canvas And Schoology In The State Of Virginia, Rosa Ann Woodley-Richard

Doctoral Dissertations and Projects

The purpose of this quantitative, causal-comparative research was to compare the differences in middle-school student achievement for grade 7 math students among three different learning management systems: Blackboard, Canvas, and Schoology. The instrument was the Virginia Grade 7 Math Standards of Learning test. The population included students in the 7th grade attending a public school in the state of Virginia. The participants were the individual school districts. The study examined the means between the pass rates of the three learning management systems to see if there was a significant difference. The independent variable included the three learning management systems: Blackboard, …

A Study Of Composite Concert Assessment Ratings Of Asboa Affiliated Ensembles From 2003 To 2019, Alyssa Perry

Student Theses and Dissertations

The purpose of this study was to examine composite ratings at Arkansas School Band and Orchestra Association (ASBOA) sanctioned concert assessments from 2003 to 2019. ASBOA divides the state into nine geographic regions to group schools for ASBOA events. All schools in a single region who choose to participate in ASBOA activities perform at concert assessment together and compete against each other at all-region auditions. In 2011, a region realignment occurred to make the number of students participating in each region in various ASBOA activities, specifically all-region and all-state bands, more equal. As a result, ASBOA adjusted the geographic regions, …

Lesbian, Gay, And Bisexual Students Experiencing Homelessness And Substance Use In The School Context: A Statewide Study, Hadass Moore, Kris De Pedro

Education Faculty Articles and Research

PURPOSE

This study explored differences between lesbian, gay, and bisexual (LGB)-housed and homeless students regarding substance use patterns on and off school grounds and the unique contribution of homelessness to substance use in school.

METHODS

Data were from the 2013-2015 California Healthy Kids Survey, a statewide survey of school protective factors and risk behaviors. A representative sample of 9th- and 11th-grade students (N = 20,337) was used. Comparisons between housed (n = 19,456) and homeless (doubled up: n = 715; acute homeless: n = 166) LGB students were conducted. We used chi-square tests to compare rates of lifetime, past-30-day, and …

Effects Of Reading Intervention On Behavior Outcomes Of Middle School Students: A Rapid Review, Davina Putz

Educational Specialist, 2020-current

Reading is one of the most valuable skills acquired in life with tremendous effects on future success in education, professional career, and daily living. While effective reading instruction continues to be focus of elementary school education, some students proceed to middle and high school stuck on an elementary grade reading level. Below grade level reading ability can result in numerous behavioral and academic struggles at the secondary level. The current study systematically investigated the body of research evaluating effects of reading interventions on behavior outcomes. Five publications met inclusionary criteria to be further analyzed using narrative synthesis and visual analysis …

Revisiting Ethnic Differences In In-Person Learning During 2021-2022, Alison Heape, Andrew Camp, Gema Zamarro

Education Reform Faculty and Graduate Students Publications

In the spring of 2020, the COVID-19 pandemic closed schools throughout the United States, forcing a shift to remote learning that lasted the rest of the academic year. In the fall of 2020, schools reopened using combinations of in-person, hybrid, and remote learning models with great geographic variability in access to in-person learning. A growing body of research shows important racial differences in the use of in-person learning during the 2020-2021 school year, with Black and Hispanic students returning to in-person learning at lower rates than white students (Camp and Zamarro, 2021; Kurmann and Lalé, 2022). This in-person learning gap …

区分问题解决专家与新手的关键因素：数据挖掘方法, Song Li Jin, Kwok Cheung Cheung, Pou Seong Sit

Chinese/English Journal of Educational Measurement and Evaluation | 教育测量与评估双语期刊

数字化环境下的问题解决能力被广泛认为是21世纪的核心技能之一。许多重要因素会影响该能力表现：有些因素针对于特定的问题并与问题解决的过程相关，而其它非问题特定的因素则与问题解决者的知识、技能、态度、信念以及学习环境相关。本研究旨在分析经合组织 (OECD) 国际学生评估项目 (PISA) 2012 年研究中基于计算机的示例问题解决测验的过程数据，挖掘能将学生区分为“问题解决专家”与“问题解决新手”的关键因素。PISA 2012 的受试学生包括来自42 个经济体的11,599 名15 岁学生。研究使用了多级逻辑回归来 分析问题解决过程和学生问卷数据；其次级数据分析采用了涉及分类和回归树的数据挖掘方法。研究结果表明有五个可区分专家与新手的重要因素。

Important Factors Discriminating Between Problem-Solving Experts And Novices: A Data Mining Approach, Song Li Jin, Kwok Cheung Cheung, Pou Seong Sit

Chinese/English Journal of Educational Measurement and Evaluation | 教育测量与评估双语期刊

Digital problem-solving competence is widely recognized as one of the core skills of the 21st century. A number of important factors influence this competence; some are task-specific pertaining to the problem-solving processes while others are non-task-specific related to knowledge, skills, attitudes and beliefs of the problem solvers, as well as the student learning environment. This study sought to determine important factors that classify student problem-solver as “high-performing expert” versus “low-performing novice”, using computer-generated log files of an exemplary digital problem task assessed in Organization for Economic Co-operation and Development (OECD)’s Programme for International Student Assessment (PISA) 2012 Study. The participants …

Positionality: The Interplay Of Space, Context And Identity, Rebecca Y. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

This article considers the way in which positionality shifts and is formed during a cross-cultural study to reveal the complexity of the insider-outsider status. As a researcher in a male-dominated game setting, I reflect on the research process and my interactions with participants to show the interplay of space, context, and identity in shaping a researcher’s status. I discuss the process of gaining access to the research site and participants, and data collection in relation to space, context, and identity. The interaction of my identities with space, and context informed my status at various moments. This interplay constructs a complex …

Ending The Status Quo: Kentucky Parents Increasingly Choose Nonpublic Education Options, Gary Houchens

School of Leadership & Professional Studies Faculty Publications

This research report documents enrollment trends for Kentucky students in nonpublic education (homeschooling and private schools) during the years 2017-2022. Using data from the Kentucky Department of Education, the analysis finds that nonpublic education participation has grown by more than 20,000 students during this time period to almost 100,000 students. This trend indicates a clear demand for more education options and policy makers should respond accordingly.

Collage As A Transdisciplinary Learning And Teaching Method In A Time Of International Crisis, James Green Dr

Arts and Design

This paper discusses the art and design method of collage and its efficacy as a trans-disciplinary learning and teaching technique that can be delivered remotely while living through a pandemic and the restrictions of a national lockdown. Collage is a method for making artworks by pasting together desperate elements, such as printed matter or digital images, to make a relatively unified image. Trans-disciplinary learning integrates perspectives from a range of different disciplines, going beyond a given students subject discipline, and should enable students to draw on these perspectives to gain a deeper understanding of their discipline. This was managed through …

Secondary Educators' Experiences Implementing Formative Assessment In Rural Southern Maryland: A Transcendental Phenomenological Qualitative Study, Bonnie J. Skinner

Doctoral Dissertations and Projects

The purpose of this phenomenological qualitative study is to describe secondary educators’ experiences in planning and implementing formative assessment in rural southern Maryland to gain an understanding of their definition and use of Assessment for Learning (AfL) in diverse classrooms. The theory guiding this study, the sociocultural learning theory, grew from the work of the psychologist Vygotsky. The sociocultural learning theory is the best worldview for this study as Vygotsky’s beliefs of parents, caregivers, peers, other mentors (teachers), and culture are responsible for the development of cognition and higher-order functions with learning occurring by interacting with other people. Four essential …

Use Of The Prepare Tool To Increase Completion Rates Of Advance Care Planning Documentation, Janella Thompson

Student Scholarly Projects

Practice Problem: One in three adults aged 55 and older admitted to the hospital lack advance care planning. Patients without advance care planning may not receive treatment that reflects their personal values, wishes, and preferences

PICOT: The PICOT question that guided this project was in adults 65 years or older (P), how does the implementation of a patient-centered web-based advance planning tool kit (I) compared to the usual practice of addressing advance care planning (C), affect the completion rate of advance care planning documentation (O) within 10 weeks (T).

Evidence: The current evidence has demonstrated that advanced care …

Implementing Tactile Learning To Aid Students Understanding Of The Bohr Model, Christin B. Monroe, Andrew B. Stein, Cindy Tolman

Journal of Science Education for Students with Disabilities

It is essential for introductory level chemistry students to understand atomic models and how atoms interact to form chemical bonds. The tactile model in this article utilizes marbles to represent subatomic particles, a cup to represent the nucleus and wooden rings to simulate the electron orbitals. These inexpensive items can be combined to construct models in which students can build foundational knowledge of atomic structure and how subatomic particles interact. Students were asked to provide feedback comparing the use of this tactile model to atomic computer simulations, videos and their textbook regarding the method they felt was most useful to …

A Winning Approach: Teaching Science Communication Skills Through Small-Group Workshops, Serena B. Gumusoglu, Maria Noterman Soulinthavong, Jennifer Barr

Higher Learning Research Communications

Objectives: Research almost always culminates in the communication of findings. Despite the necessity of grant and manuscript writing throughout academic careers, scientific trainees often receive little guided practice in written communication. To fill this gap, we designed, implemented, and evaluated a voluntary writing initiative for biomedical students at a research-intensive (R1) university in the midwestern United States called Writing Initiative in Neuroscience (WIN).

Method: WIN consisted of didactic and workshop components. The didactic component included discussions with topic-specific experts on writing grants and manuscripts for the public and for non-academic scientific careers. The workshop component consisted of small group-based peer …

Laying The Foundation For Education 4.0: Access, Value And Accountability, Jennifer Karlin, L. Eric James, Lauren Singelmann, Dan Ewert

Integrated Engineering Department Publications

The complexity of the global problems engineers are working to solve has long been discussed in both engineering and engineering education circles. The Grand Challenges for Engineering are grand because of the complexity of the challenges. While the challenges stand over a decade later, the speed at which the terms in which they are described, the shift from Industry 3.0 to Industry 4.0, has been slow. As the world becomes more deeply connected, as the internet of things becomes more commonplace in all parts of our lives, as technologies like machine learning and cyber physical systems become accessible to even …

A Multi-Decade Response To The Call For Change, Bart Johnson, Ron Ulseth, Mike Raich

Integrated Engineering Department Publications

Engineering and society have always been intertwined, especially with the accepted realization of technology's significant and rapidly increasing influence on the evolution of society. As a profession, engineering has a vital role in sustainably meeting needs and exploring opportunities that are ever changing and evolving. As societal and industry needs have evolved, engineering education itself has raised the call several times for evolving the way engineers are educated; however, the recent history of engineering education is, overall, one of missed opportunities. This was brought to a headline recently as ASEE leadership authored an article entitled “Stuck in 1955, Engineering Education …