Educational Assessment, Evaluation, and Research Commons^™

Measuring What Matters: Challenges And Opportunities In Assessing Science Proficiency, James W. Pellegrino

2009 - 2019 ACER Research Conferences

A key challenge in shaping science learning for the future will be to develop new measures of learning that take into account what it means to be proficient in science (Pellegrino, 2013). The emergent view on proficiency, grounded in learning sciences research, emphasises using and applying knowledge in the context of disciplinary practice. Referred to as knowledge-in-use, this perspective on science proficiency is a centrepiece of the United States’ National Research Council’s (NRC) Framework for K–12 Science Education (NRC, 2012), embodied in the new US national standards (NGSS Lead States, 2013) and emphasised in the recently released NRC report on …

Assessment Standards, ‘Intentional Alignment’, And Dialogic Inquiry, Claire Wyatt-Smith

2009 - 2019 ACER Research Conferences

Internationally, the policy move towards standards-aligned instruction is gaining momentum. In Australia, standards have assumed unprecedented prominence in education policy relating both to classroom practice and to teacher preparation and career progression. The move is also evident in the United States, where the lure of standards to inform improvement is clear: significant investment has been committed to longitudinal research to examine at state and district levels the desirable conditions for implementing standards, their impact on developing college- and career-ready teachers, and in turn, the impact on teacher instruction and student outcomes. Moves such as this are occurring in the absence …

Should Generic Curriculum Capabilities Be Assessed?, Rosemary Hipkins

2009 - 2019 ACER Research Conferences

Both Australia and New Zealand have recently taken up the idea of ‘key competencies’ (‘capabilities’ in the Australian national curriculum) initially proposed by the Organisation for Economic Co-operation and Development. In both countries we have made them our own by adapting them to suit our own educational contexts. People often say that these capabilities won’t be taken seriously unless they are assessed. So whether, and how, to assess them continue to be vexed questions. In this paper I argue that capabilities are more appropriately seen as changing the curriculum rather than adding to it. If we are serious about preparing …

Learning Assessments: Designing The Future, Geoff N. Masters

2009 - 2019 ACER Research Conferences

Processes for assessing student learning are undergoing fundamental transformation.This presentation will consider three developments which can be expected to shape how student learning is assessed in the future. First is fundamental change in how assessment is conceptualised and approached, with a focus on monitoring learning. Second is growing interest in the assessment of a broader range of skills and attributes than those addressed in most current assessment efforts. Third is advances in technology which are opening the door to new ways of gathering information about student learning, including through records of real-time interactions in online learning environments. In ACER’s Centre …

Creating Stealth Assessments, Val Shute, Michael Timms

2009 - 2019 ACER Research Conferences

In this workshop, participants will learn how to create stealth assessments to measure student performance during interactions within computer-based learning environments, like digital games or intelligent tutoring systems. These measures are then used to estimate various competencies, including hard-to-measure constructs like creativity, persistence, problem-solving and systems thinking. First, we will explain how evidence-centred design can be used as a theoretical approach to designing such assessments. Next, we’ll illustrate how evidence-centred design was applied in the development of stealth assessment within a particular game (using the example of Plants vs. Zombies 2). Participants will have a chance to create their …

Improving Assessment For Indigenous Students, Alison Quin

2009 - 2019 ACER Research Conferences

This workshop will introduce a teaching scenario and existing unit of work built around Indigenous culturally inclusive practices. Participants will work in small groups to devise assessment items in a scaffolded process. The development of assessment tasks will be informed by clarifying Indigenous education, the process of building relationships between schools and Indigenous communities that contribute to co-developed units of work, and principles of Indigenous culturally inclusive practice such as place-based, community-grounded and group production practices.

Assessing Young Children's Literacy And Mathematics Understandings, Collette Taylor, Joanne Mulligan, Maurice Walker, Prue Anderson, Marion Meiers

2009 - 2019 ACER Research Conferences

Current research indicates that young children are capable of developing mathematical concepts and reasoning much earlier than previously considered. Moderated by Professor Collette Tayler, this symposium on assessing young children's literacy and mathematics understandings is in three parts. Firstly the authors explore the Pattern and Structure Assessment (PASA). PASA is an early mathematical assessment interview which focuses on a range of concepts and processes and is linked with mathematical attainment in the ACER Progressive Achievement Tests in Maths (PATMaths). Secondly, the authors report on the piloting of an early-years technology-based tool called the Digital Early Reading and Mathematics Assessment (DERMA). …

New Measures For An Old Friend: A Learning Progression For Ict Literacy, Mark Wilson

2009 - 2019 ACER Research Conferences

This workshop will present new thinking and new results from the work of the Assessment and Teaching of 21st-Century Skills (ATC21S) project, on the learning progression for information and communications technology literacy — learning in digital networks. This project, initially sponsored by Cisco, Intel and Microsoft, aimed to help educators around the world enable students with the skills to succeed in future career and college goals. The workshop will be structured to show the how the development of the new ideas and measures for ICT literacy followed the logic of the assessment system developed by the Berkeley Evaluation and Assessment …

Acer Research Conference Proceedings (2015), Acer

2009 - 2019 ACER Research Conferences

It is indeed timely that Research Conference 2015 addresses the theme Learning Assessments: Designing the future. It is six years since our Research Conference considered issues in assessment, and the landscape is being significantly transformed. Not only is Australia’s school curriculum changing, but related issues of teaching quality and assessment practice are hot topics here and in many other countries. This transforming landscape includes changes in thinking about the fundamental purposes of assessment; growing demands for the assessment of a broader range of student skills and capabilities; and new technologies that allow us to gather and visualise information about student …

‘Big Five’ Challenges In School Education, Geoff N. Masters Ao

Teacher columnist – Geoff Masters

Real reform and significant progress in improving the quality and equity of Australian schooling depend on tackling our deepest and most stubborn educational challenges, writes Professor Geoff Masters AO.

Social Justice And Technocracy: Tracing The Narratives Of Inclusive Education In The United States, Scot Danforth

Education Faculty Articles and Research

Over the past two decades, the percentage of American students with disabilities educated in general classrooms with their nondisabled peers has risen by approximately fifty percent. This gradual but steady policy shift has been driven by two distinct narratives of organisational change. The social justice narrative espouses principles of equality and caring across human differences. The narrative of technocracy creates top-down, administrative pressure through hierarchical systems based on quantitative performance data. This article examines these two primary policy narratives of inclusive education in the United States, exploring the conceptual features of each and initiating an analysis of their application in …

Gaise Into The Future: Updating A Landmark Report For An Increasingly Data-Centric World, Michelle Everson, Paul Velleman, Beverly Wood, John Gabrosek, Megan Mocko, Robert Carver

Publications

Ever since its official endorsement by the American Statistical Association in 2005, the Guidelines for Assessment and Instruction in Statistics Education (GAISE) College Report has had a profound impact on the teaching of statistics. Now, a decade later, it is important to recognize the changing nature in what and how we teach our introductory statistics students. Changes in technology and assessment practices, just over the past 10 years, have made it possible to do new and exciting things in our courses, in very different ways than were envisioned by the authors of the original GAISE College Report. Further, our world …

Catholic Teachers’ Postgraduate Qualifications And Students’ End Of Schooling Outcomes: A Large Scale Queensland Based Comparative Study, Judy Smeed, Beryl Exley

eJournal of Catholic Education in Australasia

Abstract

A key feature of the current era of Australian schooling is the dominance of publically available student, school and teacher performance data. Our paper examines the intersection of data on teachers’ postgraduate qualifications and students’ end of schooling outcomes in 26 Catholic Systemic Secondary Schools and 18 Catholic Independent Secondary Schools throughout the State of Queensland. We introduce and justify taking up a new socially-just measurement model of students’ end of schooling outcomes, called the ‘Tracking and Academic Management Index’, otherwise known as ‘TAMI’. Additional analysis is focused on the outcomes of top-end students vis-à-vis all students who are …

Comprehensive School Reform: A Study On The Effectiveness Of The High Schools That Work Program In Texas, Kimberley Michelle Irving-Conaway

Theses & Dissertations

This mixed methods, sequential explanatory research study evaluated the effectiveness of a comprehensive school reform program titled High Schools That Work (HSTW). High schools were tasked with having students meet passing standards on high-stakes assessments or face sanctions. This study was necessary because, as Herman et al. (2008) said, “unfortunately, the research base on effective strategies for quickly turning around low-performing schools is sparse” (p. 4). Quantitative and qualitative data were collected to determine if schools improved after the HSTW program was implemented. First, Academic Excellence Indicator System reports on Texas HSTW program schools were downloaded from the Texas Education …

School Psychologists’ Training And Knowledge Of Tourette Syndrome, Leticia Cornejo

Educational Studies Dissertations

A web-based survey was conducted that included 97 practicing school psychologists in California. The results from the survey indicated that the majority (88%) of respondents were knowledgeable about Tourette Syndrome. Many respondents (28%) had never worked with a student with Tourette’s, 20% had at least one case, and 52% indicated that they had worked with more than two cases in their careers as school psychologists. The majority of respondents indicated that their school psychology program did not adequately train them to assess or counsel students with Tourette’s. The majority of participants also did not feel confident to work with students …

Teaching & Learning Workshops At The Center For Teaching & Learning At Thomas Jefferson University Ay15-16, Center For Teaching And Learning

Academic Commons and Scott Memorial Library Staff Papers and Presentations

No abstract provided.

One Academic Year Study Of Experiences Of One Cohort Of Graduates From A Midwestern University’S Teacher Education Program, Chris Albers

Dissertations

Teacher attrition is a growing problem within the education system in the United States. A study that tracked the experiences of graduates of a particular program was designed to gauge the program’s success. There were three research questions. The first asked, What are the first year teaching experiences of Midwestern University educator preparation program graduates? The second question was in two parts and asked, What do graduates of Midwestern University educator preparation program perceive as the strengths and weaknesses of their ability to carry out their duties as a first-year teacher? How does this change from August to May? The …

International Computer And Information Literacy Study: Icils 2013: Technical Report, Julian Fraillon, Wolfram Schulz, Tim Friedman, John Ainley, Eveline Gebhardt

ICT - Digital Literacy

The International Computer and Information Literacy Study (ICILS) studied the extent to which young people have developed computer and information literacy (CIL) to support their capacity to participate in the digital age. ICILS systematically investigated differences in CIL outcomes across the participating countries and looked at how these countries provide CIL-related education. The study also explored differences within and across countries with respect to the relationship between the outcomes of CIL education and student characteristics and school contexts. Chapter 1 of this report presents an overview of the International Computer and Information Literacy Study. Chapters 2 and 3 are concerned …

Assessing High School Science Students’ Abilities To Use Cross Cutting Literacy Skills And Scientific Argumentative Writing Skills In A Mid-Western School District, Ellen M. Karel

Masters Theses

This study sought to determine to what extent a centrally focused initiative concentrated on how to teach students to not just write, but how to think, read, and speak about real world problems in a persuasive manner based on multiple sets of data related to science concepts, increased scientific argumentative writing proficiencies among high school students. A secondary area this study attempted to explore was the correlation between the implemented processes in the initiative and high school students’ scientific argumentative writing proficiencies. The study was conducted in a mid-western high school, population 1,088, with a select group of students in …

Barriers To Computer Programming Student Success: A Quantitative Study Of Community College Students In Southwest Missouri, Tiffany D. Ford

Dissertations

Student success in computer programming courses has been a long-studied problem and computer science major retention has historically been substantially lower than other majors. The issue of retention for computer science majors has become more pronounced in two-year, open-enrollment institutions. This quantitative study, grounded in Gardner’s theory of multiple intelligences, attempted to address some of the causes of poor retention for entry-level computer science majors at two-year colleges by looking for predictors of student success in their first computer programming course. Two of the intelligences from Gardner’s (1993) theory, Logical-Mathematical and Visual-Spatial, were used along with two factors: student success …

Analyzing The Change And Development Of Simulation Self-Efficacy Among Practical Nursing Students, Moulay Abdelkarim Moukrime

Dissertations

The goal of this dissertation was to contribute to research on practical nursing students’ self-efficacy and the sources that build self-efficacy including mastery experience, vicarious learning, social persuasion, and physiological and affective states (Bandura, 1986). Specifically, the focus in this study was on students’ self-efficacy change and development through the measurement of students’ confidence in ability to engage in medical surgical simulations during the last semester of a practical nursing program. The results of this study revealed perceived self-efficacy did not change, but participants indicated an overall strong sense of efficacy to engage in medical surgical simulations. Additionally, students relied …

Tennessee Per-Pupil Expenditures In Special Education And Academic Achievement, Melanie C. Davidson

Electronic Theses and Dissertations

Federal legislation known as the No Child Left Behind act has required states to close achievement gaps. The purpose of this study was to investigate the relationships between special education spending by districts in the state of Tennessee and standardized achievement for students with disabilities.

Secondary data were obtained from the Tennessee Department of Education’s website (www.tn.gov/education). Data from each local education association were collected for the years of 2010-2014 in the areas of special education spending, special education population, and TCAP proficiency percentages in reading and math grades 3-8. Data from each local education association were collected …

What Professional Development Practices Support The Successful Integration Of Technology Within A Standards-Based Educational (Sbe) System, Richard A. Green

All Theses And Dissertations

The purpose of this quantitative study was to explore the relationship between K-12 teachers’ current technology skill level, their self-efficacy as teachers, and their attitude toward changes required to integrate 21st-century technologies into their classrooms. Twenty-five members of the teaching staff from a rural School Department in Maine participated. This study describes the following: (a) What is the relationship between teachers’ current skills/ability using technology and their attitude towards integrating 21st-century technologies in the classroom? (b) What is the relationship between teachers’ current self-efficacy towards using technology, and their capacity towards integrating 21st-century technologies in the classroom? and (c) What …

Autism Spectrum Disorders: Available Public Education Programming And The Influence Of Professional Perceptions, Heather Wilmot

All Theses And Dissertations

A mixed-methods research study was conducted in a small, rural western Maine school department. The intent of this research was to reveal what kind of programming is available to students with autism spectrum disorders (ASD) and how the availability of programming is influenced by the perceptions of educators and leaders in a public school department. In addition, the research addressed how the background and professional development of the employees impacted the availability of programming for students with ASD. Forced survey questions included quality indicators for programming for students with ASD and pre-selected follow-up focus groups questions were used.

Data analysis …

“It’S The Best Idea Ever!”: Exam For The Byod Generation., Susan J. Main Mrs, Alistair Campbell Phd

eCULTURE

This research sought to investigate the feasibility of digitising exams to improve student outcomes and was based on research suggesting that the handwriting speed of undergraduate students limits their ability to demonstrate knowledge, while poor legibility makes it difficult for the assessor to accurately judge the quality of the response. Research found that the handwriting speed of undergraduate students was equivalent to fluency data on 11-year-old schoolchildren, which is a significant concern when we consider that handwriting fluency accounted for considerable variance in writing quality and tutor marks for examination answers. This generation of students typically relies on digital technologies …

Collaboration Opportunities With The Society To Enhance Indonesian Citizenry By Furthering Higher Education With The Liberal Arts (Seiche), Judith Puncochar, Don Faust

Conference Presentations

The Society for the Enhancement of Indonesian Citizenry through Higher Education (SEICHE) is an Indonesian organization analogous to The Association for the Study of Higher Education (ASHE) in the United States. SEICHE and ASHE use similar organizational structures and promote scholarly research and debate about issues and possibilities in the advancement of higher education. SEICHE was launched on November 22, 2014, in Bandung, Indonesia, with representatives from seven universities and one non-profit organization. The SEICHE vision is to help achieve international recognition of Indonesia’s higher education system, accomplished principally through collaboration and engagement with research in curriculum, instruction, service, and …

The Effects Of Spatial Skills And Spatial Skills Training On Academic Performance In Stem Education, Gavin Duffy, Stephanie Farrell, Rachel Harding, Avril Behan, Aaron Mac Raighne, Robert Howard, Edmund Nevin, Brian Bowe

Conference papers

Spatial ability plays an important but often unacknowledged role in achieving success in science, technology, engineering and mathematical (STEM) education. Many entering STEM disciplines have low spatial skills but these can be improved through a short training course. Accompanying improvements in academic grades and retention rates have been observed by others. This presents an opportunity to enhance the first year experience (FYE) for those with poor spatial skills. In this study the spatial skills of students entering several first year programmes in science and engineering were measured. Those identified as weak visualisers were offered a spatial skills course. Spatial skill …

A Profile Of The Spatial Visualisation Abilities Of First Year Engineering And Science Students, Stephanie Farrell, Avril Behan, Gavin Duffy, Rachel Harding, Robert Howard, Aaron Mac Raighne, Edmund Nevin, Brian Bowe

Conference papers

The link between spatial visualisation skills and academic and professional achievement in STEM fields is well established. It is also widely documented that men outperform women on tests of spatial ability. This disparity puts women at a relative disadvantage for academic success in STEM disciplines. Our research explores the influence of secondary academic experience on spatial visualisation ability among first year students entering an engineering or science programme. Spatial ability was measured using two standard psychometric instruments for measurement of spatial visualization. Out of eight secondary courses considered, a single course in design and computer graphics emerged as a predictor …

Assessing The Validity And Reliability Of Dichotomous Test Results Using Item Response Theory On A Group Of First Year Engineering Students, Edmund Nevin, Avril Behan, Gavin Duffy, Stephanie Farrell, Rachel Harding, Robert Howard, Aaron Mac Raighne, Brian Bowe

Conference papers

Traditional measurement instruments employed to assess the performance of student’s studying on STEM (Science, Technology, Engineering and Mathematics) related programmes typically involve classification based on final scores. The validity and reliability of these instruments and test forms are important considerations when assessing whether a student understands content and if not, where and in what way they are struggling. The aim of this study is to examine, validate and analyse the test results of first-year engineering student’s at an Institute of Higher Learning in Ireland who took the Purdue Spatial Visualisation Test of Rotation (PSVT:R). Results obtained were analysed using the …

Numeracy Infusion Course For Higher Education (Niche), 1: Teaching Faculty How To Improve Students' Quantitative Reasoning Skills Through Cognitive Illusions, Frank Wang, Esther I. Wilder

Numeracy

We describe one of the eight units of a professional development program, the Numeracy Infusion Course for Higher Education (NICHE), which introduces research on cognition, including dual-processing theories, to university faculty. Under the dual-processing framework, System 1 (intuition) quickly proposes intuitive answers to judgment problems as they arise, while System 2 (deliberation) monitors the quality of these proposals, which it may endorse, correct, or override. We present several classic questions that demonstrate the pitfalls of overreliance on intuition without analytical thinking, then describe faculty participants’ responses to these questions and their ideas on how to apply cognitive illusion research to …