Education Commons^™

Confirmative Evaluation - A New Cipp Model, Tia Lynnece Finney

Wayne State University Dissertations

ABSTRACT

CONFIRMATIVE EVALUATION: A NEW LEVEL TO THE CIPP PROGRAM EVALUATION MODEL

By

Tia L. Finney

August 2020

Advisor: Dr. Shlomo Sawilowsky

Major: Education, Evaluation and Research

Degree: Doctor of Philosophy

Struggling trainees often require a substantial investment of time, effort, and resources from medical educators. An emergent challenge involves developing effective ways to accurately identify struggling students and better understand the primary causal factors underlying their poor performance. Identifying the potential reasons for poor performance in medical school is a key first step in developing suitable remediation plans (Artino, et al., 2010). The SOM Modified Program is a remediation …

Advancing Objectives-Oriented Evaluation With Participatory Evaluation Methodology – A Mixed Methods Study, Dustin R. Saalman

Wayne State University Dissertations

The ability to complete program evaluations of educational programming is typically restricted by the availability of resources, such as time, money and a trained evaluator. A mixed methods study was completed to explore the use of a participatory evaluation program evaluation with the use of the program objectives as an advanced organizer. Participatory evaluation is purported to increase organizational learning and promote evaluative thinking within an organization (Cousins & Whitmore, 1998). Objectives oriented evaluation is an easily understood evaluation method which provides a refined focus program outcome (Madaus & Stufflebeam, 1989). An explanatory sequential design was employed utilizing quantitative findings …

A Study Of Self-Evaluation And Estimator Error In Internal Medicine Residents And Its Influence On The Academic Achievement Of Medical Knowledge, William Morse

Wayne State University Dissertations

The main objective of this study is to address the research question—how does a sample of internal medicine residents self-evaluate and what, if any, influence does this have on their academic achievement of medical knowledge? The research builds on and extends our understanding of the self-evaluation process through the moderating effects between competency-related beliefs and academic achievement of medical knowledge. The study design is quantitative, cross-sectional survey research using a non-random sample of 58 internal medicine residents at Henry Ford Hospital in Detroit, Michigan. The primary sources for data collection included a study survey that measured competency-related beliefs as a …

Pilot Evaluation Of The Computer-Based Assessment Of Non-Cognitive Attributes Of Health Professionals (Cana-Hp), Sara Faye Maher

Wayne State University Dissertations

To meet the needs of complex and/or underserved patient populations, health care professionals must possess diverse backgrounds, qualities, and skill sets. Holistic review has been used to diversify student admissions through examination of non-cognitive attributes of health care applicants. The objective of this study was to develop a novel methodology, the computer-based assessment of non-cognitive attributes of health professionals (CANA­ HP), to effectively screen non-cognitive attributes of applicants. Three research questions were delineated; 1.) To determine the CANA-HP instrument reliability (internal consistency & interrater), 2.) To determine if the CANA-HP measured attributes of non­cognitive variables, as demonstrated by low construct …

Reliability And Validity Of An Evaluation Capacity Assessment Instrument For Principals Of Public Schools In Michigan And The Current Distribution Of Evaluation Capacity By Poverty Concentration, Christine M. Lewis

Wayne State University Dissertations

This study used Confirmatory Factor Analysis (CFA) to investigate the reliability and validity of an Evaluation Capacity Assessment Instrument (ECAI) for principals of K-12 public schools in Michigan. The ECAI was then used to investigate whether scores between principals of high poverty schools and low poverty schools were significantly different. The ECAI was administered to 121 principals across Michigan. Based on the results of the CFA, evidence of fit to the a priori model was not established but internal consistency statistics provide some reliability evidence. A Principle Components Analysis (PCA) was conducted and factor loadings were examined. Some groups of …

A Comparative Study Of Kendall-Theil Sen, Siegel Vs Quantile Regression With Outliers, Ahmad Farooqi

Wayne State University Dissertations

Researchers in social and behavioral sciences usually interested in study the relationship between a response variable Y_i and one or more independent predictors〖 X〗_i either for the purpose of explanation or prediction. Ordinary Least Square Regression is a parametric approach used to study this kind of relationship. One of the disadvantages of Ordinary Least Square is it does not fit well in the presence of outliers in the response variable Y_i or both in the response variable Y_i and the predictor variable〖 X〗_i, also if the data were sampled from a non-normal distribution. Quantile Regression, Theil-Sen regression, and the modified …

Comparison Of Readability Indices With Grades 1-5 Narrative And Expository Texts, Susan Beth Harden

Wayne State University Dissertations

The problem that exists when using one or more readability indexes to ascertain a text grade level is the varied outcomes received on any given text from readability indexes that purport to measure the same construct. This study aims to provide practitioners with data to make informed decisions regarding interchangeability of readability indexes. A total of n = 244 narrative (n = 116) and expository texts (n = 128) passages from grades 1-5 were evaluated using the following readability indexes: Flesch-Kincaid Grade Level, Fry Graph, Spache, Dale-Chall, Gunning Fog, and Smog. Fifteen (15) comparison sets were analyzed using Bland-Altman method …

Reliability And Validity Of An Arabic Translation Of Academic Self-Efficacy Scale (Ase) On Students At King Faisal University, Mohammed Fahad Almohazie

Wayne State University Dissertations

ABSTRACT

RELIABILITY AND VALIDITY OF AN ARABIC TRANSLATION OF ACADEMIC SELF-EFFICACY SCALE (ASE) ON STUDENTS AT KING FAISAL UNIVERSITY

by

MOHAMMED F. AL MOHAZIE

May 2018

Advisor: Dr. Shlomo Sawilowsky

Major: Evaluation and Research

Degree: Doctor of Philosophy

Academic self-efficacy refers to a student’s confidence in successfully performing academic tasks at a designated level. Empirical research in US students suggests academic self-efficacy is a positive predictor of university student academic performance and retention, two outcomes that need to be increased in Saudi students. Given the potential predictive ability of academic self-efficacy on student academic performance and retention, a reliable and …

Reliability And Validity Of An Arabic Version Of The Self-Determination Assessment – Internet (Sda-I), Mona Alamri Alamri

Wayne State University Dissertations

Self-determination is related to the desirable transition outcomes of an individual. It has gained an increasing amount of attention in numerous fields, including education, sociology, psychology, and other fields related to human behavior. However, there are no measures originally written in Arabic that accurately measure an individual’s self-determination. Thus, during this study, an Arabic version of the Self-Determination Assessment – Internet (SDAi) was developed from the regular English version of the assessment. The SDAi measures the cognitive, emotional, and behavioral factors related to self-determination. The purpose of this study was to develop and then test the reliability and validity of …

Reliability And Validity Of Michigan School Libraries For The 21st Century Measurement Benchmarks, Natosha Nicole Floyd

Wayne State University Dissertations

The purpose of this study was to examine the psychometric properties of the Michigan School Libraries for the 21st Century Measurement Benchmarks (SL21). The instrument consists of 19 items with three subscales: Building the 21st Century Learning Environment Subscale, Teaching for 21st Century Learning Subscale, and Leading the Way to 21st Century Learning Subscale. The sample consisted of 54 respondents who were administered the instrument in 2014 and 2015. Cronbach’s alpha for the total instrument was 0.807 (n = 19 items). Exploratory factor analysis (EFA) was used to measure construct validity. The findings derived from the EFA did not tend …

The Effects Of Various High School Scheduling Models On Student Achievement In Michigan, Russell Edward Pickell

Wayne State University Dissertations

This study reviews research and data to determine whether student achievement is affected by the high school scheduling model, and whether changes in scheduling models result in statistically significant changes in student achievement, as measured by the ACT Composite, ACT English Language Arts, and ACT Math scores. The high school scheduling models included in this study are: six period semester (SPS), five period trimester (FPT), seven period semester (VPS), standard 4x4 block (SB), and alternate day block (ADB). Supportive and non-supportive research are discussed in regard to the impact of the student scheduling model on student achievement.

Robustness And Comparative Power Of Welch-Aspin, Alexander-Govern And Yuen Tests Under Non-Normality And Variance Heteroscedasticity, Ayed A. Almoied

Wayne State University Dissertations

Classical statistical tests are used in many disciplines such as education and psychology. Such tests are based on certain assumptions (e.g., normality and homoscedasticity) that are must to be met in order to produce accurate results. Violation of such assumptions is a common problem researchers encounter, particularly when analyzing real data. When such assumptions are violated, the effectiveness and efficiency of tests to control over the probability of a Type I error and maintain a relatively level of statistical power will be substantially affected.

Alternative modern and robust statistical tests such as Yuen test for trimmed means and the Alexander-Govern …

An Evaluation Of The Predictors Of Placement In Permanent Supportive Housing In The Detroit Metropolitan Communities, Andrea King-Jimenez

Wayne State University Dissertations

This was a quantitative evaluation of the predictors in placement of permanent supportive housing using the Coordinated Assessment Model (CAM) located in the Southwest Solution Agency. The statistical methods used were chi-square, discriminant function analysis and the logistical regression. The dependent variables were housed and not housed and the independent variables were chronic, family, Service Prioritization Decision Assessment Tool, and veteran status. There were 277 cases used for this study, the statistical methods showed there was a difference between the numbers of cases housed verse not housed. The discriminant function analysis and the logistic regression indicated the independent variables family …

Student, Teacher, School, And District Determinants Of Elementary School Accountability Classification In The Detroit Metropolitan Region, Embekka Nakia Thompson

Wayne State University Dissertations

Legislators have implemented an accountability system in an effort to increase school quality for all students. Policymakers assume that rewards and sanctions, based on results from high-stakes testing, can improve student achievement. The accountability system does not consider environmental factors that impact student outcomes. This study examines student, teacher, school and district determinants and their impact on student achievement, school designation, and school ranking. Data was collected from 333 elementary (K-5) schools in 56 public school districts and 18 charter schools in three counties: Macomb, Oakland, and Wayne. Two research questions were developed for this study.The first research question examined …

Comparison Of Cox Regression And Discrete Time Survival Models, Hong Ye

Wayne State University Dissertations

A standard analysis of prostate cancer biochemical failure data is done by conducting two approaches in which risk factors or covariates are measured. Cox regression and discrete-time survival models were compared under different attributes: sample size, time periods, and parameters in the model. The person-period data was reconstructed when examining the same data in discrete-time survival model. Twenty-four numerical examples covering a variety of sample sizes, time periods, and number of parameters displayed the closeness of Cox regression and discrete-time survival methods in situations typical of the cancer study.

Objective Statistics For The Measurement Of Agreement, Elizabeth Perkin Moen

Wayne State University Dissertations

The prevailing method for biomedical device interchangeability is a scatterplot of means and differences bounded by levels of agreement (Bland-Altman Plot) which results in subjective judgments made in the discernment of differences in data distribution and fit. The purpose of this study was to test four statistics - the Kolmogorov-Smirnov test, and three fit indices (NFI, GFI, & CFI) in order to identify a more objective statistic for device interchangeability. The Kolmogorov-Smirnov test proved incompatible with the data structure. The Normed Fit Index (NFI) proved most sensitive to correlation shifts where the Goodness of Fit Index (GFI) and Comparative Fit …

Development Of The Acute Decompensated Heart Failure Risk Model For Emergency Room Resident Training, Zora Cvetkovski-Injic

Wayne State University Dissertations

The purpose of this study was to characterize patients hospitalized with acute decompensated heart failure with and without low systolic blood pressure using exploratory factor analysis (EFA). Direct and surrogate measurements were measured. The aim was to use EFA for data reduction to elicit a parsimonious set of factors summarizing the relationships between variables by measuring intercorrelations of the clinical variables collected as part of standard care, and abstracted from electronic medical records.

A better understanding of the characteristics and outcomes of the target group could potentially lead to individualized treatment modalities tailored to effectively and economically improve care. Patients …

Self-Directed Learning And The Development Of Self-Efficacy In Basic Writing, Amy Ann Latawiec

Wayne State University Dissertations

This dissertation examines and analyzes the work of two sections of basic writing over the course of one semester. I explore relevant research in Writing Studies, Cognitive Psychology, and Educational Psychology to build a framework within which to discuss pedagogical strategies implemented to support student’s self-directed learning behaviors and to positively affect their efficacy beliefs. Through an analysis of students’ written work, I determined whether and how this pedagogy facilitated students’ articulation of efficacy beliefs as evidence through the language of their reflective writing assignments. Analysis of the data suggests three major arguments: first, that while self-efficacy is a complex …

Meta-Synthesis Of Sampling Methods Using Coding Strategies And Social Media For Deaf And Hard Of Hearing Nursing Students, Timberly Robinson Robinson

Wayne State University Dissertations

In 2010, the birth of an aggressive strategy began to emerge aimed to increase the number of Deaf and Hard of Hearing (D/HH) individuals in professional healthcare fields. Employers and higher education admission officials desired more research. Data revealed that the field of nursing had a higher percentage of Deaf and Hard of Hearing professionals than other areas. Yet, there is a gap in the literature related to in-depth accounts about the learning experience and clinical training of D/HH nursing students. A phenomenological study was conducted to examine nursing experiences (homogenous sampling) compared to other clinical and non-clinical D/HH majors …

A New Logic Model For Change, Zsa-Zsa Lashawn-Marie Booker

Wayne State University Dissertations

Logic models are defined as visual diagrams that help to explain the theory of change for a program. Over the years the logic model has become a common tool for educational programs who seek to apply for and obtain grant funding. However, the limitations of the logic model make it ineffective at managing evaluations. This study is a retrospective cohort design. The three main goals of this study are to (1) research logic model limitations and adapt a revised logic model that could effectively evaluate an educational program, (2) test both the original and revised logic models on an educational …

The Impact Of Classroom Observations And Collaborative Feedback On Evaluation Of Teacher Performance, Based On The Danielson Framework For Teaching, Christine L. Hofer

Wayne State University Dissertations

ABSTRACT

THE IMPACT OF CLASSROOM OBSERVATIONS AND COLLABORATIVE FEEDBACK ON EVALUATION OF TEACHER PERFORMANCE, BASED ON THE DANIELSON FRAMEWORK FOR TEACHING

by

CHRISTINE L. HOFER

December 2016

Advisor: Dr. Thomas Edwards

Major: Education

Degree: Doctor of Philosophy

Teacher evaluation systems in Michigan are undergoing major reforms driven by recent legislation at both the federal and state levels. Multiple teacher observations, as well as student achievement data, are now required to be a major indicator of teacher effectiveness for evaluative purposes. The reformed system is high-stakes, as employment decisions such as layoffs and termination rest squarely on evaluation results. Implementation has …

Robustness And Power Comparison Of The Mood-Westenberg And Siegel-Tukey Tests, Linda Candy Lowenstein

Wayne State University Dissertations

The author examined how, in the context of experimental design, one might become aware of the Behrens-Fisher problem (heteroscedasticity) in order to apply an approximate solution, such as the Yuen's statistic (1974). It was expected that both the Mood-Westenberg dispersion test (1948) and the Siegel-Tukey test (1960) would remain robust with respect to Type I and Type II error properties (and associated power levels) for detecting variance changes when their assumptions of equal means was slightly violated (i.e., the Behrens-Fisher problem). With the use of Monte Carlo Simulations, the author reviewed 34,606 permutations composed of interactions between various sample sizes, …

The Impact Of Academic Optimism On Student Achievement In Five Middle Schools, Rita Hickey Teague

Wayne State University Dissertations

This study addressed the research question: To what extent do the measures of academic optimism predict academic achievement? The study was conducted in two suburban, adjacent school districts with 102 middle school teachers from five middle schools participating.

A non-experimental, multiple regression research design was used as no treatment or intervention was provided. Data collection tools were a revision of three surveys that purport to measure the three components of academic optimism, and school level outcomes from the Michigan Education Assessment Program and the Measures of Academic Progress. Using Google Forms, data were collected electronically at faculty meetings with submission …

An Evaluation Of Wayne State University's Educational Evaluation And Research Program, Willie L. White Ii

Wayne State University Dissertations

This is a mixed-methods evaluation of the Educational Evaluation and Research program at Wayne State University. The process of evaluation involved determining the efficacy of triangulating qualitative and quantitative methods of evaluation in order to access the EER program's goal acquisition. The process of evaluation commenced with a qualitative method of interviewing faculty members and was triangulated quantitatively with a likert scale survey that was modified from Wayne State University's Student Evaluation of Teaching Survey (SETS). The Student Evaluation of the Educational Evaluation and Research Program (SEEERP) was designed to measure graduate students' perspectives of EER goals and objectives acquisition. …

Influence Of Target Population Misspecification On Employee Perceptions At A, Joe Lee Smith

Wayne State University Dissertations

Numerous researchers have conducted qualitative and quantitative studies examining employee perceptions related to changes in their work environment based upon management/top-down (deductive) communication of vision, mission, and envisioned organization goals Hofstede, Neuijen, Daval, Ohayv, & Sanders (1990), but research on the influence of subgroup/identity types on workforce perception is sparse Dutton, Dukerich, & Harquail (1994). Data on subgroup identification with the mission and strategic goals envisioned by management/administration is limited. Also limited is knowledge of the influence they have over their members, which places management at a disadvantage in planning strategic organization objectives Albert & Whetten, (1985). These subgroups have …

The Reliability And Validity Of The Thin Slice Technique: Observational Research On Video Recorded Medical Interactions, Tanina Suzanne Foster

Wayne State University Dissertations

The Reliability and Validity of the Thin Slice Technique: Observational Research on Video Recorded Medical Interactions

Introduction: Observational research using the thin slice technique has been routinely incorporated in observational research methods, however there is limited evidence supporting use of this technique compared to full interaction coding. The purpose of this study was to determine if this technique could be reliability coded, if ratings are consistent between the first, second and third slice, and if they are indeed representative of full interactions.

Methods: Three 30-second thin slices were sampled from the beginning, middle and end of a full-length video-recorded …

A Case Study Of The Impact Of A Sytematic Evaluation Process In A Graduate Medical Education Residency Program, Heidi T. Kromrei

Wayne State University Dissertations

The Accreditation Council for Graduate Medical Education has charged institutions that sponsor accredited Graduate Medical Education programs (residency and fellowship specialty programs) with overseeing implementation of mandatory annual program evaluation efforts to ensure compliance with regulatory requirements. Physicians receive scant, if any, training in program evaluation methodology. Human Performance Technology (HPT) offers models suitable for residency program evaluation as well as trained evaluators who are experts in evaluation. Leaders in the field of HPT have called for empirical studies to examine the impact of HPT models in a variety of contexts.

This single case study examined the impact of using …

Validity Of The Educator Evaluation Instrument In The State Of West Virginia, Carla Howe

Wayne State University Dissertations

In the state of West Virginia, the educator evaluation system was implemented in 2010 as part of a comprehensive system of support to increase teacher effectiveness and student learning. As part of the system, the Educator Evaluation Instrument was developed to measure teachers' effectiveness. This study was conducted to determine whether the Educator Evaluation Instrument was valid for use in measuring effectiveness.

A hierarchical confirmatory factor analysis (HCFA) was conducted on the scores from the demonstration year. The data were not normal, nor was good model fit established based on the current model. Because good model fit could not be …

Building Local Skills: The Multitrait-Multimethod Matrix In Practice, Anna Clara Gersh

Wayne State University Dissertations

The potential of expanding the evaluative skills of site-based practitioners is examined in a variety of educational enterprises by introducing a procedure to determine evidence for construct validity of measurement tools. The expectations of administrators of educational enterprises, including school day principals, administrators of after school and extended day programs, camps and other instructional settings to effectively collect and manage data is growing. Research skills are an important part of both accountability and improvement efforts which are frequently tied to funding. The multitrait-multimethod matrix (MTMM) (Campbell & Fiske, 1959) combined with the Sawilowsky I test (Sawilowsky, 2002) may provide a …

Assessment Practices Of Students With Learning Disabilities In Lebanese Private Schools: A National Survey, Rasha Elhage

Wayne State University Dissertations

Education is intended to provide diverse students with the skills and competencies needed to enhance their lives (Salvia, Ysseldyke & Bolt, 2011). This includes assessment practices that enable teachers to identify students' current level of skills, their strength and weaknesses, target instruction at student's personal level, monitor student learning and progress and plan and conduct adjustments in instruction, and evaluate the extent to which students have met instructional goals. This study intended to discover, describe, and evaluate the assessment practices of teachers and administrators working with students with learning disabilities in Lebanese private schools via the Context, Input, Process and …