Educational Assessment, Evaluation, and Research Commons^™

Communicating About Computational Thinking: Understanding Affordances Of Portfolios For Assessing High School Students’ Computational Thinking And Participation Practices, Deborah A. Fields, Debora Lui, Yasmin Kafai, Gayithri Jayathirtha, Justice Walker, Mia Shaw

Instructional Technology and Learning Sciences Faculty Publications

Background and Context: While assessment of computational thinking concepts, practices, and perspectives is at the forefront of K-12 CS education, supporting student communication about computation has received relatively little attention.

Objective: To examine the usability of process-based portfolios for capturing students’ communication about their computational practices regarding the process of making electronic textile projects.

Method: We examined the portfolios of 248 high school students in 15 introductory CS classrooms from largely underserved communities, using a formal rubric (top-down) to code computational communication and an open-coding scheme (bottom-up) to identify computational practices described.

Findings: Students demonstrated stronger …

Case Conceptualization As An Alternative To Educationally Related Mental Health Assessments, Michael R. Hass, Zack Maupin, Michael Doria

Education Faculty Articles and Research

School psychologists play an essential role in the provision of school-based mental health services yet continue to spend the majority of their time conducting psychoeducational assessments. In California, changes in law regarding the provision of mental health services have increased the tension around the role of school psychologists and led to models for determining the need for mental health services that are inefficient and present a potential barrier to students receiving services in a timely manner. The paper proposes case conceptualization as a more useful and efficient approach, than traditional assessment processes for determining students’ mental needs and writing goals.

Community College Full-Time Faculty Members’ Perceptions Of Course Grading, April C. Zawlocki

Graduate Research Theses & Dissertations

This study examined community college full-time faculty members’ perceptions of grading and reporting student learning via course grades using a mixed methods approach. The research questions that guided this study asked about the differences that exist in community college full-time faculty members’ perceptions of grading by different academic subdivision, by having or not having a degree in education, and by gender. Additionally, the study examined how perceptions of the purpose of grades differ among faculty members. The study also sought to find out which criteria were used to determine course grades and what experiences faculty members attribute to learning how …

Learning From Failures: Lessons From Unsuccessful Grading Reform Initiatives, Thomas R. Guskey

Educational, School, and Counseling Psychology Faculty Publications

This article describes accounts of grading reform initiatives that while well-intentioned, met with staunch opposition and eventually were abandoned. The implementation strategies employed by the leaders of these reform initiative are explored, along with reasons these strategies failed to result in meaningful and enduring change. Alternative grading reform strategies with supporting evidence are offered, justification for their use explained, and new directions for grading reform initiatives recommended.

Perspectives On Professional Development On Common Core Standards For English Language Arts, Martin W. Jones

Walden Dissertations and Doctoral Studies

Middle school teachers at a rural site in a western state have faced problems in implementing Common Core State Standards (CCSS) for English Language Arts (ELA), as well as in resolving achievement gaps in ELA between regular education students, special education (SPED) students, English language learners (ELL), and at-risk students. The purpose of this case study was to obtain teachers’ and school leaders’ perspectives on how CCSS for ELA can be used to enhance learning for all populations of students using the Universal Design for Learning (UDL) principles. The CCSS for ELA and UDL comprised the frameworks employed in this …

Informing Makerspace Outcomes Through A Linguistic Analysis Of Written And Video-Recorded Project Assessments, Kevin M. Oliver, Jennifer K. Houchins, Robert L. Moore, Chuang Wang

STEMPS Faculty Publications

A growing body of research focuses on what outcomes to assess in makerspaces, and appropriate formats for capturing those outcomes (e.g. reflections, surveys, and port-folios). Linguistic analysis as a data mining technique holds promise for revealing different dimensions of learning exhibited by students in makerspaces. In this study, student reflections on makerspace projects were gathered in 2 formats over 2 years: private written assessments captured in the 3D GameLab gamification platform, and semi-public video-recorded assessments posted in the more social FlipGrid platform. Transcripts of student assessments were analyzed using Linguistic Inquiry Word Count (LIWC) to generate 4 summary variables thought …

Towards The Development Of A Framework For The Inclusion Of Peer Assessment In Irish Medical Science Undergraduate Education, Mary Mcgrath

Doctoral

Assessment of learning is integral to all programmes in Higher Education (HE). While one purpose of assessment is to confirm achievement of standards, another is to support student learning. When educational programmes adopt an assessment strategy that is inclusive of ‘assessment for learning’ the role of assessment supporting learning can be expanded and sustained more effectively. Peer Assessment (PA) is an example of an assessment approach that can support and enrich student learning. The advantages and benefits of PA are strongly established in the published literature.

This research study aimed to investigate and develop a framework for the inclusion of …

Support For Teacher Candidates And The Edtpa, Dia Gary, Dylan Thomas, Joseph Miller

Journal of Global Education and Research

Equipping new teachers in today’s society is a noble and challenging task. Of late, many additional licensing standards for teachers create additional responsibilities for universities that provide pedagogy, knowledge, and content for teacher candidates. Identification of best practices that support teacher candidates on the Educative Teacher Performance Assessment (edTPA) is of interest to many teacher education programs. The purpose of this research was to analyze whether incorporating a mock edTPA had beneficial results on the passage of the edTPA. This study spanned fourteen ten-week quarters and included 688 teacher candidates who were preparing to become certificated teachers at a university …

Evaluating The Effectiveness Of Thompson’S (2105) Survey Of Language Usage And Spanish Of Heritage Learners Placement Program Exam At The High-School Level, Elizabeth Villanueva

Doctoral Dissertations

Language placement test serve with multiple purposes to measure the language abilities and performance of language learners to place them in the most appropriate language course with curriculum that best meets the language needs. There is the need to assess appropriately the Spanish heritage language learners’ (SHLLs) abilities at the high-school level. Therefore, this mixed-method study investigated how Thompson’s (2015) modified Yes or No 10-question Survey of Language Usage (Survey) and the three part Spanish for Heritage Language Learners Placement Program Exam (Placement Test) could be used effectively with three Spanish class levels of HLLs at the high-school level to …

Data And Assessment Management In Collegiate Recreation, Jeana Carow

Graduate Theses and Dissertations

Collegiate recreation programs and centers typically provide traditional programming space in addition to a range of physical activity spaces and resources, as a valuable part of the student experience. The external pressures of identifying and communicating departmental value and impact on the campus community has resulted in collegiate recreation departments’ use of data to communicate the effectiveness and impact of their work. The purpose of the study was to identify the data collection and assessment management practices of collegiate recreation departments, particularly focusing on the organization of data and assessment strategies as well as data collection, storage, reporting, analyzing, and …

A Psychologist’S Perspective For Coordinating Interdisciplinary Courses, Amanda L. Almond

Publications and Research

Developing a student evaluation for interdisciplinary teaching revealed a clearer goal for interdisciplinary course assessment. This chapter summarizes how interdisciplinary course assessment is a cooperative and reflexive process. Using professional judgment and a working group of peers, interdisciplinary courses maintain their integrity through regular reviews. A reflection on experiences with team-teaching, guest lecturing, and learning-communities is also included. Best practices for interdisciplinary course maintenance and concepts of validity are applied to the debate between evaluation and assessment methods. By fostering transparency, accountability, and peer-led critiques, interdisciplinary learning objectives within courses are sustained each semester. Recognizing concerns regarding evaluation, flexible approaches …

When You Can’T R.I.O.T., R.I.O.: Tele-Assessment For School Psychologists, Michael R. Hass, Brian P. Leung

Education Faculty Articles and Research

The acronym R.I.O.T., record review, interview, observation, and test, is a well-known tool for conceptualizing a comprehensive assessment. With COVID-19 and the need to provide school psychological services virtually, it is important to reconsider R.I.O.T. in light of the limitations of virtual assessment. We describe the limitations of virtual assessment and argue that in spite of these barriers, the first three elements of R.I.O.T., record review, interviews, and observations, when used systematically, can provide useful comprehensive assessment data. Specific recommendations are provided for implementing assessment virtually.

Oer Project: Bsba Capstone Refresh Materials, Monika Hudson

USF OER Faculty Grant Projects

Slideshow presented by Dr. Monika Hudson for USF's Open Access Week, 2020.

The Role Of Test Making In Learning Process, Dilbar Xasanova, Hugh Cartwright Cheonan Assistant Professor

The Light of Islam

Almost always with the word "test" a question comes to mind with four or fve possible answers. Students usually think of multiple-choice questions (MCQs) when they are told to get tested. Testing means testing students' knowledge using different types of question forms. The proposed article examines the academic performance test, which was developed by the administration of the Uzbekistan English Teachers Association (UZTEA) for Year 2 students of lyceums and colleges. The purpose of this study is to discuss test constructs and propose a more reliable version of them if the reason for the unsatisfactory results depends on the evaluation …

Education Reform And The Transition To Standards-Based Grading In The United States, Lexa Mahr

Master's Theses & Capstone Projects

This literature review provides a history of education reform in the United States along with the basis for a transition to standards-based grading. The educational system is a complex network of intertwining parts that work together with the goal to mold a child into a valued member of society. Continual passage of educational laws has helped develop education reform within the United States. Adoption of federal and state laws helps create consistency amongst schools. Grades are held with high regard and deserve accurate representation within an unbiased grading system. Many components are encompassed in the transition to a standards-based grading …

Preschool Assessment, Christy Faber

Master's Theses & Capstone Projects

Assessments in early childhood are being addressed by educational leaders, administrators and teachers to meet the needs of preschoolers and ensure that these young children are school-ready. A school improvement plan was created using research-based methods. This school improvement plan looks at the need to create a developmentally appropriate preschool assessment report based on state standards with age appropriate objectives outlining specific benchmark skills to show teachers and parents an accurate picture of how children are progressing and growing. The focus of the literature review is based on the need for preschool assessments, types of assessments that are developmentally appropriate …

Probing For Bias: Comparing Populations Using Item Response Curves, Paul J. Walter, Edward Nuhfer, Crisel Suarez

Numeracy

We introduce an approach for making a quantitative comparison of the item response curves (IRCs) of any two populations on a multiple-choice test instrument. In this study, we employ simulated and actual data. We apply our approach to a dataset of 12,187 participants on the 25-item Science Literacy Concept Inventory (SLCI), which includes ample demographic data of the participants. Prior comparisons of the IRCs of different populations addressed only two populations and were made by visual inspection. Our approach allows for quickly comparing the IRCs for many pairs of populations to identify those items where substantial differences exist. For each …

Estimating School-Level Achievement In Belize, Betty Jean Usher-Tate

College of Education and Human Sciences: Dissertations, Theses, and Student Research

This dissertation consists of five chapters: introduction, literature review, methodology, results, and discussion with final thoughts. The research design of this dissertation study attended to structures, cultures, and characteristics associated with, and specific to, the Belize education system. The processes for data collection and types of analyses were appropriate, yielded meaningful results, and served as a segue for national application. The Belize Educator Survey was developed to capture the educators’ voices and illuminate their relationship to educational achievement in Belize. The Belize Educator Survey was piloted and revised with direct input from educators and experts who work in the Belize …

Breaking Up The Grade, Thomas R. Guskey

Educational, School, and Counseling Psychology Faculty Publications

To make grading more meaningful, course grades should reflect a range of distinct criteria that make up student learning.

Eight Essential Principles For Improving Grading, Susan Brookhart, Thomas R. Guskey, Jay Mctighe, Dylan Wiliam

Educational, School, and Counseling Psychology Faculty Publications

Done well, grading can play a key role in a balanced district assessment system.

Training Design Enhancement Through Training Evaluation: Effects On Training Transfer, Elham Arabi

UNLV Theses, Dissertations, Professional Papers, and Capstones

With recurring changes in the society, in terms of economy and technology, companies need to ensure that their human capital is aligned with these transformations in order to sustain in a competitive industry. To date, organizations view training as a requirement to increase their employees’ efficiencies and productivity, and reduce their attrition rates. Despite investment in employee training, research shows a lack of transfer of learning to job and as a result minimal return on investment. This issue is due to many barriers, but can be addressed by learning professionals. Poor learning design and training evaluation practices have been identified …

Measuring Numeracy: Validity And The Programme For The International Assessment Of Adult Competencies (Piaac), Samuel L. Tunstall

Numeracy

A tension raised in recent scholarship is that between numeracy as a social practice and numeracy as a functional skill set. Such frameworks for conceptualizing numeracy pose a challenge to assessment because what individuals do with numeracy is not the same as what individuals can do (or express) in an assessment setting. This study builds on work related to numeracy assessment through a validity examination of a portion of a well-known assessment: the OECD’s Programme for the International Assessment of Adult Competencies (PIAAC). In following a path set out by standards for assessment, I ask: What does the PIAAC numeracy …

Assessment Of Children As Having A Strong Sense Of Identity In Early Childhood Education And Care: Literature Review, Jen Jackson, Kate Noble, Danielle Anzai, Pru Mitchell, Dan Cloney

Early Childhood Education

This review focuses on the Outcome: Children have a strong sense of identity. Its purpose is to equip early childhood professionals with the knowledge to identify and assess children’s progress towards this Outcome in all early childhood settings. Central to the child’s development of a sense of identity is their personal identity. This includes awareness of self, summed up in the statement: ‘I know who I am’, and includes the constructs of self-expression and self-awareness. A strong sense of identity requires that a child also appreciates who they are, summed up in the statement: ‘I value who I am’. The …

The Primary Dilemma: Determining And Overcoming Barriers To A Focus On Prekindergarten Through Grade 3, Sarah Turner Lukas, Sarah Turner Lukas

Dissertations

In 2017, reporters of national data in regard to reading ability among American, fourth grade students indicated low proficiency and achievement gaps. These results were also consistent within the state and district under study. There is a need to increase the focus on prekindergarten and primary grade education taking place prior to the grades in which federally mandated, accountability assessments for student learning take place. The purpose of this study is to determine the potential barriers to a paradigm shift at the state and district levels from a focus on state-assessed grade levels to prekindergarten and primary grade levels. The …

Self-Determination Theory And Faculty Behavior: A Quantitative Study Of Faculty Leaders' Use Of Assessment Evidence, Joseph D. Levy

Dissertations

Despite assessment of student learning being essential work in higher education, a number of institutions have noted faculty could more effectively be using assessment results (Jankowski et al., 2018; Kuh et al., 2015; Metzler & Kurz, 2019; Suskie, 2014). This study applied Self-Determination Theory (SDT) as a theoretical framework to provide context for faculty behavior associated with assessment actions (Fuller et al., 2016; Ryan & Deci, 2000). Mostly quantitative data were collected via electronic survey of faculty program leaders at a single institution, National Louis University (NLU). Results indicated a significant and positive relationship suggesting an increase in meeting the …

Evidence-Informed Programming In Student Affairs: A Mixed Methods Study Examining Behaviors, Perceptions, And Barriers Related To The Use Of Theory And Research In Program Development, Andrea M. Pope

Dissertations, 2020-current

For decades, professional organizations and leaders in the field of student affairs have called for student affairs professionals to engage in evidence-informed programming (EIP). EIP refers to the use of theory and empirical research to build programs intended to impact specific student learning or development outcomes. The benefits of EIP range from increasing the likelihood that newly developed programs will “work” to increasing the efficiency of the assessment process and facilitating the use of assessment results for program improvement. Despite the many calls for EIP, there is concern that EIP in student affairs is rare; however, empirical research on professionals’ …

Dominican Nursing Students’ Self-Reported Mastery Of International Nursing Competencies, Wady Ramirez

Dissertations

The Dominican health care system has many challenges to meet the legal and the population’s health care expectations (La Hoz, 2018; United States Agency for International Development, 2013). The purpose of this study was to evaluate how well Dominican nursing students, who have completed their internship but not yet defended their final career project, assessed themselves against a set of international nursing competencies. In addition, the purpose was to explore any differences in their self-assessments based on whether these students completed their internship at public health care centers or not. Finally, the goal was to determine to what extent do …

The Dark Side Of Assessment Literacy: Avoiding The Perils Of Accountability, Thomas R. Guskey

Educational, School, and Counseling Psychology Faculty Publications

Educational measurement and evaluation experts generally agree that increasing stakeholders’ assessment literacy will yield a variety of positive benefits, especially broadening the range of assessment formats teachers use to measure students’ mastery of high level, more cognitively complex learning outcomes. But in the context of education accountability as currently structured in American schools, such efforts also may lead teachers to become more sophisticated in test preparation activities and to narrow both their instruction and classroom assessment practices specifically to enhance students’ performance on prescribed, annual high-stakes accountability assessments. This article explains why that is so, describes the process by which …

Saudi Arabian Science Teachers And Formative Assessment In The Gender Segregated Male School System, Khalid Abdullah Kariri

Dissertations

Formative assessment is a key pedagogical tool that allows teachers to make instructional assessments in real time for the improvement of student learning. It gives students the opportunity to provide evidence of their learning thus giving teachers assurances that their hard work is not in vain. Formative assessment is commonly practiced in Western schools, while most Saudi science classroom teachers rely on end-of-unit summative assessment. A review of the literature reveals that very little research has been done related to Saudi teachers’ use of formative assessment in science classrooms. The goal of this dissertation was to address this absence of …

Analyzing The Retention Of Knowledge Among General Chemistry Students, James T. Kingsepp

USF Tampa Graduate Theses and Dissertations

General chemistry is a survey course meant to form a foundation for students’ to build upon as they progress into chemistry sub-disciplines. The effectiveness of this foundation rests heavily on students’ abilities to retain or relearn concepts from the course. This study seeks to investigate the utility of a measure of general chemistry students’ knowledge retention across the time period of one semester. This approach was enacted in five different courses of either first or second semester general chemistry. The results showed that one-quarter to one-half of students exhibited a high rate of retention across the semester. A smaller group, …