Education Commons^™

The Meta-Analysis Of Wisconsin Card Sorting Test Reliability, Naila N. Al Kharusi, Hussain Al Kharousi, Mai Helmy

An-Najah University Journal for Research - B (Humanities)

The study aimed at meta-analysis of the reliability coefficients of Wisconsin Card Sorting Test scores to find out the typical estimate value of the reliability coefficient of test scores, and to reveal the various factors that may contribute to the high or low value of the reliability coefficient. A total of (36) studies were subjected to meta-analysis of reliability, where the number of reliability coefficients calculated were (206) reliability coefficients out of (16) indicators of the Wisconsin Card Sorting Test. The current study focused on (98) reliability coefficients calculated by the method of test-retest reliability for six indicators. The results …

Validity And Reliability Of The Persian Version Of The Suboptimal Health Status Questionnaire Among University Staff In Iran, Erfan Ayubi, Salman Khazaei, Shiva Borzouei, Ali R. Soltanian, Samereh Ghelichkhani, Fatemeh Karbin, Yuxiang Yan, Manshu Song, Cuihong Tian, Wei Zhang, Jing Sun, Wei Wang

Research outputs 2022 to 2026

Background: Suboptimal Health Status Questionnaire-25 (SHSQ-25) is an established tool for measuring a precision health state between health and illness. The present study aims to assess the validity and reliability of a Persian version of SHSQ-25 (P-SHSQ-25) in a university staff Iranian population. Methods: A sample of 316 academic and supporting staff (163 males, age range from 23 to 64 years old) from Hamadan University of Medical Sciences, Hamadan, Iran was recruited in this population-based cross-sectional study with a questionnaire validation from Apri1 to October 2022. Forward-backward translation method was performed for the SHSQ-25 translation from English to Persian. Internal …

Assessing The Reliability Of Stryd 27 For Variable Speed Running, Kathryn A. Lavin, Dustin W. Davis, Matahn Blank, Tashari A. Carballo, Elias M. Malek, Thea Sweder, Eric Lambert, Nicole Argueta, Marcus M. Lawrence, Jeffrey Cowley, James W. Navalta

International Journal of Exercise Science: Conference Proceedings

Wearable technology is beneficial when it comes to tracking and optimizing performance. The Stryd 27 is a wearable footpod marketed as being more responsive in measuring power during running than the previous version (Stryd 25). However, the reliability of this newer device to return consistent values has not been determined. PURPOSE: This study aimed to observe whether Stryd 27 gives reliable metrics during variable speed running. METHODS: Sixteen participants (N = 16; 50% female; height = 174.1 ± 8.1 cm; mass = 73.0 ± 12.4 kg) were recruited, each equipped with two Stryd 27 footpods (updated to the same …

The Reliability Of Maslach Burnout Inventory In Arab Studies: Reliability Generalization Meta-Analyses Study, Saud Shaish Alenezi Phd

International Journal for Research in Education

The current study aimed to identify the mean effect size of the alpha coefficient for the overall reliability of the MBI scale, and for the sub-dimensions; Emotional exhaustion, depersonalization, and lack of personal achievement through the analysis of (32) studies in which the inclusion criteria were met. The results of the research indicated that the mean alpha coefficient across the studies (n = 32) for overall reliability and dimensions; Emotional exhaustion, depersonalization, and lack of personal achievement were 0.83, 0.83, 0.78, and 0.77, respectively. It indicated that the mean alpha coefficient of the MBI scale varied significantly according to the …

Estimating Reliability On Tests With Dichotomous And Polytomous Items According To The Classical Theory Of Measurement And Generalizability Theory: A Comparative Study, Abdal Hadi Al-Gharaibh, Ahmad Bani Ahmad

Jordan Journal of Applied Science-Humanities Series

This study aimed to compare reliability coefficients on tests with dichotomous and polytomous items according to classical theory of measurement and generalizability theory, to achieve this, a compound ability test was constructed in mathematics in two forms: Dichotomous and Polytomous by using five estimation methods, which as follows: (Alpha {α}, Theta{θ}, Raju, {G}, and {φ}), Furthermore, in order to achieve the study objectives, the test was applied on two forms, each test consists of 30 items to measure students mathematical ability on the field of numbers, Algebra, Geometries and Statistics, to 300 of the sample study. the reliability coefficients of …

Examining Dimensionality And Validity Of The Academic Integrity Survey Instrument, Zhidong Zhang, Luis Garcia

Teaching and Learning Faculty Publications and Presentations

Dimensional reduction is one of the methods to ensure the quality of a questionnaire. This study examined two methods to reduce the dimension of the questionnaire: multidimensional scaling (MDS) and exploratory factor analysis (EFA). The questionnaire, Awareness of Academic Dishonesty consists of 30 questions. Participants included 110 college students. Multidimensional scaling analysis reduced the multidimensions to essentially two dimensions. The exploratory factor analysis reduced the multidimensions to three dimensions. MDS allowed the researchers to evaluate the questionnaire items by looking at the similarities of these data points. EFA provided an alternative thought about the construct of the questionnaire.

Is It Actually Reliable? Examining Statistical Methods For Inter-Rater Reliability Of A Rubric In Graduate Education, Brent J. Goertzen, Kaley Klaus

Leadership Faculty Publications

When evaluating student learning, educators often employ scoring rubrics, for which quality can be determined through evaluating validity and reliability. This article discusses the norming process utilized in a graduate organizational leadership program for a capstone scoring rubric. Concepts of validity and reliability are discussed, as is the development of a scoring rubric. Various statistical measures of inter-rater reliability are presented and effectiveness of those measures are discussed. Our findings indicated that inter-rater reliability can be achieved in graduate scoring rubrics, though the strength of reliability varies substantially based on the selected statistical measure. Recommendations for determining validity and measuring …

Assessing The Reliability Of The Five-In-20 Classroom Observation Tool, Danielle Buechlein

Masters Theses

Effective classroom management practices are crucial to fostering a positive learning environment, student achievement, and student social-emotional development, as well as teacher job satisfaction. The Five-in-20 Classroom Observation Tool (FCOT) was developed from the 21 evidence-based strategies identified by Simonsen and colleagues (2008) to assess and support teachers’ classroom management; however, its psychometric properties are unknown. The present study looks at the interobserver agreement of the FCOT, specifically at how consistent observers’ ratings are with each other and how consistent observers’ ratings are with a standard coded copy of the FCOT. Forty-four participants, using the FCOT, rated the same 20-minute …

Reliability Of The Stryd Accelerometer On An Incline And Decline, Wesley Ziegler, Jeffrey Cowley

International Journal of Exercise Science: Conference Proceedings

Modern technology has evolved to include various health and fitness trackers to help the general population become more aware of their physical wellbeing. However, the reliability of these technologies is not well-established. PURPOSE: Determine the reliability of Stryd accelerometer during trail running. METHODS: Seventeen participants (25 ± 9 years; F = 7) completed a trail run. Each participant was fitted with one Stryd accelerometer (model 25; Stryd, Boulder, CO 80301) on the shoe of each foot, and both devices were started simultaneously. The participant then did a self-paced out and back run on a moderate difficulty hiking trail. The trail …

An Innovative Method For Teachers To Formatively Assess Writing Online, Dr Sandy Heldsinger, Stephen Humphry

2021-2030 ACER Research Conferences

Assessment is an integral component of effective teaching and a teacher’s professional judgement influences all routine aspects of their work. In the last 20 years, there has been considerable work internationally to support teachers in using assessment to improve student learning. However, there is a pressing issue that impedes teacher professional judgement being exploited to its full potential. The issue relates to teacher assessments in the context of extended performances such as essays and arises from the complexity of obtaining reliable or consistent teacher assessments of students’ work. Literature published in the United States, England and Australia details evidence of …

The Set Of Test Tasks Assessing Special Physical Fitness Of 17-Year-Old Soccer Players, Andrzej Szwarc, Bartosz Dolanski, Patrycja Lipinska

Baltic Journal of Health and Physical Activity

Background: The objective of the research was to work out the set of test tasks assessing special physical fitness of 17-year-old soccer players. Material/Methods: At first, on the basis of the analysis of the said issue in literature and the authors’ own coaching experience, 13 test tasks were selected. Next, reliability and accuracy of the chosen test tasks were examined on the group of 16 athletes from the Centre of Sports training for youth in Gdansk. The reliability was assessed by the test-re-test method and the accuracy according to a criterion of internal accuracy tests. Except for one task, all …

Determination Of Maximal Oxygen Uptake Through A New Basketball-Specific Field Test, Ersen Adsiz, Gulbin Rudarli Nalcakan, S. Rana Varol, Faik Vural

Baltic Journal of Health and Physical Activity

The aim of this study was to develop a new basketball-specific field test (BSFT) that determines the maximal oxygen uptake indirectly and to determine the reliability and validity of the test for measuring aerobic power. An aerobic endurance-based test totalling 1.5 miles on a basketball court including side steps and running was designed to be completed as quickly as possible. 15-year-old, male basketball players’ maximal oxygen uptake (VO2max) levels measured directly as the gold standard laboratory test. Laboratory test results showed a significant negative correlation between the VO2max level and the duration of the BSFT (r = -0.705, p = …

Athlete Self-Efficacy Scale: Development And Psychometric Properties, Çalık Veli Koçak

Baltic Journal of Health and Physical Activity

Background: It is important that psychological structures have an impact on athlete performance, such as self-efficacy can be measured consistently. The aim of this study is to develop a measurement tool with psychometric properties that can measure the self-efficacy beliefs of athletes. Material and methods: The participants of the study consisted of 325 athletes (age 21.6 ± 4.2) who actively pursue sports in various sports branches in Turkey. In the validity and reliability analysis of the scale, exploratory and confirmatory factor analyzes were used. Cronbach’s Alpha reliability coefficient value of the total scale is .88. Results: The validity and reliability …

Validation Of The Lithuanian Version Of The Perceived Parental Success Standards In Sport Scale (Ppsss), Enrika Kromerova-Dubinskiene, Saulius Sukys

Baltic Journal of Health and Physical Activity

Background: The recently developed Perceived Parental Success Standards Scale (PPSSS) provides a new opportunity to understand better how parents perceive success in sport from the young athlete’s perspective. The aim of the present study was to adapt and validate the Lithuanian-language version of the PPSSS. Material and methods: The study included 587 adolescents (mean age = 14.3, SD = 1.11) from different sport teams in Lithuania. Confirmatory factor analysis (CFA) and additionally exploratory Pearson’s correlation and regression analysis were performed to test discriminant, concurrent, and predictive validity. Results: Exploratory structural equation modelling provided an acceptable fit of a two-factor model …

The Reliability And Intensity Dependence Of Maximum Mouth Pressure During Bench Press Exercise In Healthy Athletes, Dusan Blazek, Anna Pisz, Ondrej Pecha, Petr Kubový

Baltic Journal of Health and Physical Activity

Background: Respiratory muscle strength is an important indicator of human health and sport condition. The aim of this study was to assess the test/retest reliability of mouth pressure (MP) measured by a created pressure sensor mouthpiece in the sitting position and during bench pressing at various intensities. Material and methods: 23 healthy athletes were measured in 3 separate sessions for MP in the sitting position and during flat bench press at an intensity of one repetition maximum (1RM), 3 x 90% of 1RM, 8 x 80% of 1RM and 12 x 60% of 1RM. Results: The measurement device showed acceptable …

Content, Construct, And Criterion Validity, Reliability, And Objectivity For Aquatic Readiness Assessment For Brazilian Children, Nadia C. Valentini, Keila R G Pereira, Glauber C. Nobre

International Journal of Aquatic Research and Education

The Aquatic Readiness Assessment (ARA) is an assessment instrument for measuring children's aquatic readiness. The objective of the study was to translate the English version into Portuguese and to investigate the content, construct, and criterion validity as well as the reliability and rater objectivity of the ARA for Brazilian children. Twenty-three professionals and 464 children, newborn to 13 years-old participated in the study. We found strong content (94% to 100% of judges’ agreement) and criterion validity, internal consistency (α from .96 to .97), and inter-rater objectivity (ICC from .81 to .98), and test-retest reliability (ICC from .94 to .98). Appropriate …

Validation Of Objective Structured Clinical Examination (Osce) Based On The Occupational Therapy Practice Framework (Otpf): A Pilot Study, Marzieh Pashmdarfard, Afsoon Hassani Mehraban, Narges Shafaroodi, Kamran Soltani Arabshahi, Soroor Parvizy, Akram Azad, Samaneh Karamali Esmaeili

Journal of Occupational Therapy Education

Fieldwork education is an integral part of the educational process in occupational therapy and assessing student competency at the end of fieldwork is important. The aim of this study was to design and conduct an Objective Structured Clinical Examination (OSCE) based on the Occupational Therapy Practice Framework (OTPF) for occupational therapy students on Level II fieldwork in Iran. A seven-station OSCE was designed and conducted with 13 students. Face and content validity of the exam scenarios and grading checklists was assessed via faculty review. The correlation between scores from each station and total OSCE scores were obtained to assess construct …

The Characteristics Of Chemistry Test Items On Nationally-Standardized School Examination In Yogyakarta City, Ummul Karimah, Heri Retnawati, Deni Hadiana, Pujiastuti Pujiastuti, Eri Yusron

REID (Research and Evaluation in Education)

This study aimed to describe the characteristics of chemistry test items in Nationally Standardized School Examination or Ujian Sekolah Berstandar Nasional (USBN), consisted of discrimination index, difficulty index, and question reliability. The data were collected by using documentation of the answers of 194 students. The type of research was descriptive exploratory with quantitative and qualitative approaches. Quantitative data analysis was performed by using the classical test theory approach and item response theory with 1 logistical parameter. Meanwhile, qualitative data analysis was conducted to describe the items categorized as difficult and bad categories. The results show that, according to classical test …

Observation Of Field Practice Rubric: Establishing Content Validity And Reliability, Ruchi Bhatnagar, Carla L. Tanguay, Caroline Sullivan, Joyce E. Many

Georgia Educational Researcher

Most teacher education assessments are criticized for lacking validity and reliability. This study describes the process of developing the Observation of Field Performance rubric to assess initial teacher candidates’ classroom performance and establishing the content validity as well as reliability of the rubric. A panel of content area experts determined that 10 out of 12 items of the rubric were essential and the CVR was above the acceptable range for all 12 items, indicating that the rubric had a strong content validity. Additionally, the analysis of instructors’ ratings on the rubric showed that the rubric had good level of internal …

A Psychometric Evaluation Of The Best In Class Adherence And Competence Scale, Katrina A. Markowicz

Theses and Dissertations

It is critical to utilize treatment integrity instruments to support the evaluation of evidence-based programs in early classroom contexts. However, in the early childhood field, guidelines for collecting treatment integrity data are underdeveloped. Consequently, most treatment integrity instruments employed in the field solely assess adherence, vary in design features and have little psychometric evidence supporting their use. As such, this represents a gap in the field that might slow efforts to implement evidence-based programs. The current study examines the score reliability and validity of an observational treatment integrity instrument (The BEST in CLASS Adherence and Competence Scale [BiCACS]; Sutherland et …

Quality Analysis Of Teacher-Made Tests In Financial Accounting Subject At Vocational High Schools, Heni Mulyani, Heraeni Tanuatmodjo, Rangga Iskandar

Jurnal Pendidikan Vokasi

Assessment of student learning outcomes needs to be done using tests that meet the criteria for quality tests. This study aims to determine the quality of teacher-made tests on financial accounting subjects in Vocational High Schools. This research is descriptive research with a quantitative approach. Data collected are questions made by 32 teachers, answer sheets from 689 Accounting students. The validity of objective and essay tests using product-moment correlation. Reliability of the objective test using the KR20 formula, while the essay test using Alpha formula. Difficulty level and distinguishing power of objective tests using Anates 4. Difficulty level and distinguishing …

Pengujian Validitas, Reliabilitas, Dan Relevansi Norma Penilaian "Dyer Tennis Test" Terhadap Tingkat Keterampilan Bermain Tenis, Guntur Guntur, Ngatman Ngatman, Sridadi Sridadi, Danang Pujo Broto, Duwi Kurnianto Pambudi

Jurnal Keolahragaan

Penelitian ini bertujuan untuk (1) menguji tingkat validitas norma penilaian Dyer Tennis Test terhadap tingkat keterampilan bermain tenis mahasiswa Fakultas Ilmu Keolahragaan Universitas Negeri Yogyakarta (FIK UNY); (2) menguji tingkat reliabilitas norma penilaian Dyer Tennis Test terhadap tingkat kete-rampilan bermain tenis; (3) menguji relevansi norma penilaian Dyer Tennis Test terhadap tingkat keterampilan bermain tenis. Penelitian deskriptif ini menggunakan metode survey. Pengumpulan data menggunakan tes dan pengukuran. Berdasarkan hasil penelitian, dapat disimpulkan bahwa: (1) Koefisien validitas Dyer Tennis Test (X) dan Ranking Hasil Pertandingan ½ kompetisi Putera (Y) = -0,968 dan Hasil Pertandingan ½ kompetisi Putera Puteri (Y) = -0,997. Dengan …

Comparison Of Methods For Detecting Anomalous Behavior On Large-Scale Computer-Based Exams Based On Response Time And Responses, Deni Hadiana, Bahrul Hayat, Burhanuddin Tola

REID (Research and Evaluation in Education)

This study aims to determine the anomalous index (indeks anomali or IA) that considers both response time and responses and compares it with response time effort (RTE) or rapid guessing (tebakan cepat or TC) on various thresholds. Response time and responses from 732 examinees are in natural science subjects consist of 40 multiple choice items with four answer choices. Response time and responses are analyzed to obtain descriptive statistics related to them, calculate the TC and IA index using two methods of the threshold, the first method (M1) is a visualization of identification, and the second method (M2) …

Marshak And Shakespeare: The Question Of New Content In Translation, Zilola Alibekova

Mental Enlightenment Scientific-Methodological Journal

The article is devoted to the problems of comparative literary criticism, in particular the question of Shakespeare's sonnets in the works of S.Ya.Marshak. Marshak always remained faithful to the classical traditions of Pushkin's times and protected these traditions with irresistible tenacity. The logic of the figurative structure, the deliberate adherence to verse forms, consecrated by centuries of practice, laconicism, the concentration of distinct thought and the experience settled in the word are the features of Marshakov's art. One way or another, Marshak is organically united in all his work and must be viewed through the prism of its amazingly multifaceted, …

Maximizing Data Quality And Shortening Survey Time: Three-Form Planned Missing Data Survey Design, E. Whitney G. Moore, Kyle M. Lang, Elizabeth M. Grandfield

Kinesiology, Health and Sport Studies

Simulation studies have shown the three-form planned missing data design efficiently collects high quality data while reducing participant burden. This methodology is rarely used in sport and exercise psychology. Therefore, we conducted a re-sampling study with existing sport and exercise psychology survey data to test how three-form planned missing data survey design implemented with different item distribution approaches effect constructs’ internal measurement structure and validity. Results supported the efficacy of the three-form planned missing data survey design for cross-sectional data collection. Sample sizes of at least 300 (i.e., 100 per form) are recommended for having unbiased parameter estimates. It is …

Accreditation By Design: Construction Of An Instrument To Measure Teacher Candidates’ Perceptions Of Preparedness To Meet Intasc Standards, Michael Floren, Chelsie Hess, Valerie J.H. Sherman, Nancy M. Sileo

Journal of Educational Research and Innovation

The InTASC Candidate Self-Perception Instrument (ICSPI) is an innovative, high-quality educational measurement tool designed to support the assessment and accreditation efforts of a wide variety of educator preparation programs (EPP). The procedures used for the creation and refining of items for the ICSPI are presented, including empirical estimates of reliability from a trial implementation across three semesters. Evidence of instrument validity is discussed, while observed reliability estimates were found to be above acceptable levels for all subgroups. The ICSPI can be used to evaluate EPPs’ course offerings and requirements, align course outcomes with educational standards, provide data to advise programs …

Pilot Evaluation Of The Computer-Based Assessment Of Non-Cognitive Attributes Of Health Professionals (Cana-Hp), Sara Faye Maher

Wayne State University Dissertations

To meet the needs of complex and/or underserved patient populations, health care professionals must possess diverse backgrounds, qualities, and skill sets. Holistic review has been used to diversify student admissions through examination of non-cognitive attributes of health care applicants. The objective of this study was to develop a novel methodology, the computer-based assessment of non-cognitive attributes of health professionals (CANA­ HP), to effectively screen non-cognitive attributes of applicants. Three research questions were delineated; 1.) To determine the CANA-HP instrument reliability (internal consistency & interrater), 2.) To determine if the CANA-HP measured attributes of non­cognitive variables, as demonstrated by low construct …

Psychometric Developments Of The Ace-Iq: Understanding The Trauma History Of Latine Immigrants, Jorli K. Swingen Dr.

Theses and Dissertations

Adverse childhood experiences (ACEs) have a profound effect on an individual’s physical and mental health. The World Health Organization has recently updated the ACE questionnaire so it could be used with international populations. The Adverse Childhood Experiences-International Questionnaire (ACE-IQ), has not been translated to Spanish or used with Latine immigrants.

This study translated the ACE-IQ into Spanish, evaluated the psychometric properties of the questionnaire, and collected data on 184 four adult English Language Learners in Chesterfield County, VA.

There is evidence of internal consistency for the ACE-IQ as a whole (α = .908) and within subscales. Three factors were identified …

The Reliability Of Single-Item Assessments, Sijun Zhang

Legacy Theses & Dissertations (2009 - 2024)

Single-item assessments have become more popular recently in distinct areas, even though there is no consensus about whether they are sufficiently reliable. Researchers have developed methods to estimate the reliability of single-item assessments, some are based on factor analysis (method FA), correction for attenuation (method CA), and others employ Molenaar and Sijtsma’s theory (method MS), coefficient λ6 (method λ6), or latent class model (method LCRC). However, no empirical study has investigated which method estimates the reliability of single-item assessment most precisely. This study investigated this question via a simulation study. To represent assessments as found in practice, the simulation study …

Evaluation Of Modern Missing Data Handling Methods For Coefficient Alpha, Katerina Matysova

College of Education and Human Sciences: Dissertations, Theses, and Student Research

When assessing a certain characteristic or trait using a multiple item measure, quality of that measure can be assessed by examining the reliability. To avoid multiple time points, reliability can be represented by internal consistency, which is most commonly calculated using Cronbach’s coefficient alpha. Almost every time human participants are involved in research, there is missing data involved. Missing data means that even though complete data were expected to be collected, some data are missing. Missing data can follow different patterns as well as be the result of different mechanisms. One traditional way to deal with missing data is listwise …