Psychology Commons^™

A Novel Examination Of None-Of-The-Above As It Influences Examinee Item Responses, Kathryn N. Thompson

Dissertations, 2020-current

It is imperative to collect validity evidence prior to interpreting and using test scores. During the process of collecting validity evidence, test developers should consider whether test scores are contaminated by sources of extraneous information. This is referred to as construct irrelevant variance, or the “degree to which test scores are affected by processes that are extraneous to the test’s intended purpose” (AERA et al., 2014, p. 12). One possible source of construct irrelevant variance is violating item-writing guidelines, such as to “avoid the use of none-of-the-above” in multiple-choice items (Rodriguez, 2016, p. 268).

Numerous studies have been conducted with …

Item Development And Psychometric Assessment Of A Pilots' Well-Being Scale: A Study On Indonesian Commercial Pilots, Patricia Yora Wenas, Angela Oktavia Suryani, Laura Fransisca Sudarnoto

International Conference on Assessment and Learning (ICAL)

Flight success and accidents rely on the pilot's condition. To date, there is no psychometrically tested instrument to measure the psychological well-being of pilots. This study aims to develop items for measuring the well-being of commercial pilots in Indonesia. The constructs, dimensions, and indicators were formulated through literature reviews and interviews with pilots about their well-being. We develop well-being as a state in which a person feels positive, healthy, prosperous, comfortable, and valued in their work, able to control and contribute to the social environment. The instrument conveyed six dimensions, namely (1) positive emotion, (2) health, (3) competence, (4) recognition, …

Validation Of The Child And Adult Social Support Scale (Casss) Which Measures Social Support In The Indonesian Version, Endah Mastuti, Fajrianthi, Fitri Andriani

International Conference on Assessment and Learning (ICAL)

Social support has an important role, so that students in online learning can reduce the various problems they face. Social support here comes from parents, teachers, classmates, close friends, and the school itself as an institution. Based on this, it is necessary to do research related to social support to get an overview of the support from whom students need in online learning. Furthermore, this can be used as input to intervene in the problems faced by students. To conduct this research, it is necessary to have a measuring tool to conduct social support research. One of the comprehensive measuring …

Creating A Short, Public-Domain Version Of The Cpai-2: Using An Algorithmic Approach To Develop Public-Domain Measures Of Indigenous Personality Traits, Mukhunth Raghavan

USF Tampa Graduate Theses and Dissertations

In this study we aimed to create a short, public-domain analogue of the Cross-Cultural (Chinese) Personality Assessment Inventory (CPAI-2; F. M. Cheung et al., 1996). Emic (culture-specific) traits measured by the CPAI-2 are purportedly specific to the Chinese culture and argued to not be fully captured by the consensus Big Five personality trait taxonomy. Research suggests that CPAI-2 traits may have unique predictive power, especially in non-Western contexts. However, research has been hampered by several limitations of the measure. The inventory is proprietary and long, with 341 items forming 28 scales and four factors. Cross-cultural personality research would benefit from …

Bridging Psychometric And Cognitive Models Of (General) Intelligence: An Investigation Based On Process Overlap Theory, Han Hao

CGU Theses & Dissertations

Human intelligence has been scientifically investigated as a psychological construct for over a century but there has not been a universally accepted definition or theory. One cause of this problem is that traditional theories attempt to explain the robust findings in cognitive ability testing, such as the positive manifold, from two different perspectives: psychometric or cognitive. Both approaches have their own limitations and are sometimes incompatible with each other. Therefore, contemporary theories of intelligence have been developed to provide a more unified perspective by combining both types of approaches, allowing the psychometric structure of cognitive abilities to be represented and …

Development Of An Explainability Scale To Evaluate Explainable Artificial Intelligence (Xai) Methods, Stephen Mccarthy

Dissertations

Explainable Artificial Intelligence (XAI) is an area of research that develops methods and techniques to make the results of artificial intelligence understood by humans. In recent years, there has been an increased demand for XAI methods to be developed due to model architectures getting more complicated and government regulations requiring transparency in machine learning models. With this increased demand has come an increased need for instruments to evaluate XAI methods. However, there are few, if none, valid and reliable instruments that take into account human opinion and cover all aspects of explainability. Therefore, this study developed an objective, human-centred questionnaire …

Multiracial Individuals And Educational Testing, Karen Alexander

The Nebraska Educator: A Student-Led Journal

A literature review focused on quantitative measures and methods regarding multiracial individuals and educational testing revealed that multiracial individuals are uniquely different than monoracial individuals in terms of their racial identity and these unique identities interact with test scores. Until recently, this uniqueness has been ignored by institutions and within the field of educational testing. The uniqueness of multiracial identity should be taken into consideration when using test measures to make decisions for selection and when comparing group outcomes. The review provides a brief picture regarding the history of categorization of multiracial individuals and current research which connects the multiracial …

Can Ratings Of Item Location Enhance Statistical Item Parameter Estimation? Extending The Feasibility Of Unfolding Irt Models, Michael Mckenna

Dissertations

Research and development of modern psychometric methods such as item response theory have drastically changed the way we understand and carry out the measurement of psychological constructs. Despite this, there has been relatively little adoption by psychological researchers to incorporate these methods into their research. While multiple explanations are surely valid, one oft stated reason is the large sample size requirements of these methods. The sample size requirements of item response theory are needed so that effective estimation of item parameters can be carried out. In an attempt to make these modern measurement methods more accessible and feasible to psychological …

Psychometric Properties Of A Working Memory Span Task, Juan M. Alzate Vanegas

Honors Undergraduate Theses

The intent of this thesis is to examine the psychometric properties of a complex span task (CST) developed to measure working memory capacity (WMC) using measurements obtained from a sample of 68 undergraduate students at the University of Central Florida. The Grocery List Task (GLT) promises several design improvements over traditional CSTs in a prior study about individual differences in WMC and distraction effects on driving performance, and it offers potential benefits for studying WMC as well as the serial-position effect. Currently, the working memory system is composed of domain-general memorial storage processes and information-processing, which involves the use of …

The Trouble With Test Banks, Harvey Richman, Molly Hrezo

Perspectives In Learning

We compared the psychometrics of quiz questions randomly selected from a test bank with the psychometrics of quiz questions the instructor had selected from the bank for quality and modified (if necessary). On multiple psychometric indices, the instructor selected/modified questions were superior to questions randomly selected from the test bank. Most notably, when compared with instructor written/modified questions, randomly selected bank questions were nearly 6.5 times more likely to contain a distractor that drew more responses than the correct answer. Details and implications are discussed.

Identifying Examinees Who Possess Distinct And Reliable Subscores When Added Value Is Lacking For The Total Sample, Joseph A. Rios

Doctoral Dissertations

Research has demonstrated that although subdomain information may provide no added value beyond the total score, in some contexts such information is of utility to particular demographic subgroups (Sinharay & Haberman, 2014). However, it is argued that the utility of reporting subscores for an individual should not be based on one’s manifest characteristics (e.g., gender or ethnicity), but rather on individual needs for diagnostic information, which is driven by multidimensionality in subdomain scores. To improve the validity of diagnostic information, this study proposed the use of Mahalanobis Distance and H^T indices to assess whether an individual’s data significantly departs …

Development And Validation Of A State-Based Measure Of Emotion Dysregulation: The State Difficulties In Emotion Regulation Scale (S-Ders), Jason M. Lavender, Matthew T. Tull, David Dilillo, Terri Messman-Moore, Kim L. Gratz

Department of Psychology: Faculty Publications

Existing measures of emotion dysregulation typically assess dispositional tendencies and are therefore not well suited for study designs that require repeated assessments over brief intervals. The aim of this study was to develop and validate a state-based multidimensional measure of emotion dysregulation. Psychometric properties of the State Difficulties in Emotion Regulation Scale (S-DERS) were examined in a large representative community sample of young adult women drawn from four sites (N = 484). Exploratory factor analysis suggested a four-factor solution, with results supporting the internal consistency, construct validity, and predictive validity of the total scale and the four subscales: Nonacceptance (i.e., …

Cross-Cultural Adaptation, Validation And Reliability Of The Brazilian Version Of The Richmond Compulsive Buying Scale, Priscilla Leite, Bernard Range, Monika Kukar-Kinney, Nancy Ridgway, Kent Monroe, Rodolfo Ribas Jr., J. Landeira-Fernandez, Antonio Egidio Nardi, Adriana Silva

Nancy Ridgway

Objective: To present the process of transcultural adaptation of the Richmond Compulsive Buying Scale to Brazilian Portuguese. Methods: For the semantic adaptation step, the scale was translated to Portuguese and then back-translated to English by two professional translators and one psychologist, without any communication between them. The scale was then applied to 20 participants from the general population for language adjustments. For the construct validation step, an exploratory factor analysis was performed, using the scree plot test, principal component analysis for factor extraction, and Varimax rotation. For convergent validity, the correlation matrix was analyzed through Pearson’s coefficient. Results: The scale …

Measuring The Outliers: An Introduction To Out-Of-Level Testing With High-Achieving Students, Karen Rambo-Hernandez, Russell Warne

Russell T Warne

Out-of-level testing is an underused strategy for addressing the needs of students who score in the extremes, and when used wisely, it could provide educators with a much more accurate picture of what students know. Out-of-level testing has been shown to be an effective assessment strategy with high-achieving students; however, out-of-level testing has not been shown to work well with low-achieving students. This article provides a brief history of out-of-level testing, along with guidelines for using it.

Inventory Of Cognitive Distortions: Validation Of A Measure Of Cognitive Distortions Using A Community Sample, Michael B. Roberts

PCOM Psychology Dissertations

The purpose of this study was to examine and evaluate further the psychometric properties of a self-report inventory of cognitive distortions using a nonclinical, community sample. A group of 474 individuals were contacted via the social networking site, Facebook, and through a college list-serve and were asked to complete multiple measures and also to send the link to other individuals, thus utilizing a snowball sample. The measures used included the Inventory of Cognitive Distortions (ICD), Dysfunctional Attitude Scale (DAS), Perceived Stress Scale (PSS), and a brief questionnaire to collect demographic information on each participant. Results revealed positive psychometric properties for …

Exploring The Various Interpretations Of "Test Bias", Russell Warne, Myeongsun Yoon, Chris Price

Russell T Warne

Test bias is a hotly debated topic in society, especially as it relates to diverse groups of examinees who often score low on standardized tests. However, the phrase “test bias” has a multitude of interpretations that many people are not aware of. In this article, we explain five different meanings of “test bias” and summarize the empirical and theoretical evidence related to each interpretation. The five meanings are as follows: (a) mean group differences, (b) differential predictive validity, (c) differential item functioning, (d) differing factor structures of tests, and (e) unequal consequences of test use for various groups. We explain …

The Structure Of Child And Adolescent Aggression: Confirmatory Factor Analysis Of A Brief Peer Conflict Scale, Justin Russell

University of New Orleans Theses and Dissertations

The importance of simultaneous consideration of forms and functions in youth measures of aggressive behavior is well established. Competing models have presented these highly interrelated constructs as either independent (e.g., reactive or overt) or paired factors (e.g., reactive and overt). The current study examines these models in the context of assessing the viability of a new self-report measure, the Peer Conflict Scale – 20 Item Version. Confirmatory factor analyses were conducted on PCS 20 responses from 1,048 school-age youth living in the Gulf Coast region. Both models significantly improved upon one or two-factor alternatives, and demonstrated partial invariance across gender …

An Alternative To Cronbach's Alpha: An L-Moment-Based Measure Of Internal-Consistency Reliablilty, Todd C. Headrick, Yanyan Sheng

Todd Christopher Headrick

Data sets in the social and behavioral sciences are often small or heavy-tailed. Previous studies have demonstrated that small samples or leptokurtic distributions adversely affect the performance of Cronbach’s coefficient alpha. To address these concerns, we propose an alternative estimator of reliability based on L-comoments. The empirical results of this study demonstrate that when sample sizes are small and distributions are heavy-tailed that the proposed coefficient L-alpha has substantial advantages over the conventional Cronbach estimator of reliability in terms of relative bias and relative standard error.

Component Numeracy Skills And Decision Making, Saima Ghazal

Dissertations, Master's Theses and Master's Reports - Open

Numeracy—i.e., one’s practical understanding of mathematics in context—is one of the strongest predictors of people’s general decision making skill, independent of other cognitive abilities (e.g., intelligence, working memory, attentional control). Despite notable scientific progress on the nature of numeracy and decision making, the cognitive and decision sciences have yet to investigate individual differences in numeracy components (e.g., algebra versus probability). In this dissertation, I report on my efforts to develop new measurement technology and quantitative models of cognitive and decision skills. Analyses include the first known investigations of the relations between the major adult component numeracy skills and general decision …

Using Above-Level Testing To Track Growth In Academic Achievement In Gifted Students, Russell Warne

Russell T Warne

Above-level testing is the practice of administering aptitude or academic achievement tests that are designed for typical students in higher grades or older age-groups to gifted or high-achieving students. Although widely accepted in gifted education, above-level testing has not been subject to careful psychometric scrutiny. In this study, I examine reliability data, growth trajectories, distributions, and group differences of above-level test scores obtained from the Iowa Tests of Basic Skills and Iowa Tests of Educational Development. Two hundred twenty-four middle school students participated in this study. All participants were tested at least 1 time for an overall total of 435 …

Cross-Cultural Adaptation, Validation And Reliability Of The Brazilian Version Of The Richmond Compulsive Buying Scale, Priscilla Leite, Bernard Range, Monika Kukar-Kinney, Nancy Ridgway, Kent Monroe, Rodolfo Ribas Jr., J. Landeira-Fernandez, Antonio Egidio Nardi, Adriana Silva

Marketing Faculty Publications

Objective: To present the process of transcultural adaptation of the Richmond Compulsive Buying Scale to Brazilian Portuguese.

Methods: For the semantic adaptation step, the scale was translated to Portuguese and then back-translated to English by two professional translators and one psychologist, without any communication between them. The scale was then applied to 20 participants from the general population for language adjustments. For the construct validation step, an exploratory factor analysis was performed, using the scree plot test, principal component analysis for factor extraction, and Varimax rotation. For convergent validity, the correlation matrix was analyzed through Pearson’s coefficient.

Results: The scale …

A Preliminary Investigation Of The Validity Of Time-Based Measures Of Sustained Attention For Children, Michael R. Kulfan

Antioch University Full-Text Dissertations & Theses

This study is a preliminary investigation of the validity of using time-based measures to quantify sustained attention in children ages 6-12. Problems with sustained attention negatively affect childhood learning and development. The prevalence of disorders known to impact sustained attention performance continue to rise in the United States. Currently, commercially available, objective measures of sustained attention use normative comparisons that provide limited information about the effect such problems have on child performance in natural settings. We reviewed test data from 290 charts of children ages 6-12 referred for neuropsychological evaluation. The Test of Everyday Attention for Children (TEA-Ch) is an …

Bully/Victim Power Inventory: Measuring The Power Imbalance In The Bully/Victim Relationship, Marybeth Plonkey-Lehto

Electronic Theses and Dissertations

The empirical study of the power imbalance in the bully/victim relationship has impeded research synthesis, and the need for a quantitative measure of this key component has been well established in the literature. Lack of differentiation between victimization with and without power imbalance has been cited as a possible cause for imprecise measurement. Increased precision in bully victimization measurement is needed to accurately inform research investigating psychosocial health, treatment and positive outcomes, in addition to prevention and intervention programs. Therefore, the purpose of this dissertation was the initial development and validation of the Bully/Victim Power Inventory aimed at differentiating perceived …

An Introduction To Item Response Theory For Health Behavior Researchers, Russell Warne

Russell T Warne

OBJECTIVE:

To introduce item response theory (IRT) to health behavior researchers by contrasting it with classical test theory and providing an example of IRT in health behavior.

METHOD:

Demonstrate IRT by fitting the 2PL model to substance-use survey data from the Adolescent Health Risk Behavior questionnaire (n=1343 adolescents).

RESULTS:

An IRT 2PL model can produce viable substance use scores that differentiate different levels of substance use, resulting in improved precision and specificity at the respondent level.

CONCLUSION:

IRT is a viable option for health researchers who want to produce high-quality scores for unidimensional constructs. The results from our example-although not …

A Reliability Generalization Of The Overexcitability Questionnaire-Two (Oeqii), Russell Warne

Russell T Warne

Reliability generalization (RG) is a meta-analysis that combines and synthesizes reliability coefficients from different studies to ascertain the average observed reliability across studies. An RG study was conducted on previously reported data from 16 samples of the Overexcitability Questionnaire–Two (OEQII) with a combined N of 5,275. Cronbach’s alpha was found to be consistently higher on all OEQII subscales when scale variance was high and the sample consisted of adults. Sample size, gender composition of the sample, number of items from the subscale used, and location of sample (United States or a different county) had varying effects on observed alpha levels …

An Investigation Of Measurement Invariance Across Genders On The Overexcitability Questionnaire-Two (Oeqii), Russell Warne

Russell T Warne

The Overexcitability Questionnaire–Two (OEQII) is a quantitative instrument for assessing overexcitabilities as they are described in Dabrowski’s theory of positive disintegration. This article uses multigroup confirmatory factor analysis to examine the measurement invariance of OEQII scores across genders. Results indicate that raw OEQII scores cannot be compared across genders. Caution should be used in interpreting OEQII scores.

The Utility And Feasibility Of Metric Calibration For Basic Psychological Research, Etienne Lebel

Electronic Thesis and Dissertation Repository

Inspired by the history of the development of instruments in the physical sciences, and by past psychology giants, the following dissertation aimed to advance basic psychological science by investigating the metric calibration of psychological instruments. The over-arching goal of the dissertation was to demonstrate that it is both useful and feasible to calibrate the metric of psychological instruments so as to render their metrics non-arbitrary. Concerning utility, a conceptual analysis was executed delineating four categories of proposed benefits of non-arbitrary metrics including (a) help in the interpretation of data, (b) facilitation of construct validity research, (c) contribution to theory development, …

A Multigroup Analysis Of Reintegrative Shaming Theory: An Application To Drunk Driving Offenses, Elizabeth J. Dansie

All Graduate Theses and Dissertations, Spring 1920 to Summer 2023

A restorative justice alternative to crime prevention termed reintegrative shaming theory by Braithwaite has seen increased attention as an alternative to retributive justice, although empirical investigations of its efficacy are limited. The purpose of the present study was to test confirmatory measurement and structural models of reintegrative shaming theory in order to assess the underlying theoretical model and the application of this theory in response to drunk driving offenses. Nine latent constructs were included in these models: reintegration, stigmatization, perceived fairness, self esteem, shame-guilt, embarrassment-exposure, unresolved shame, offender responsibility, and family support.

Multigroup structural equation modeling was used to assess …

Factors Moderating The Association Between Multiple Rating Sources Of Geriatric Depression: Self, Informant, And Physician, Daniel J. Hatch

All Graduate Theses and Dissertations, Spring 1920 to Summer 2023

Late-life depression is a major public health concern, associated with poor health outcomes, including doubling of dementia risk. Psychiatric evaluation is impractical in large epidemiological studies, which instead typically rely on self/informant reports, which are subject to various biases (stigma, recall). Few studies have addressed level of agreement between sources. This study examined associations between these sources and assessed whether subject and informant variables moderated these associations. In a population-based study of dementia in Cache County, Utah (2002-5), 1,480 subjects completed an in-depth clinical assessment (CA). Major depression was assessed via the self-report Patient Health Questionnaire-9 (PHQ-9) and informant-rated Neuropsychiatric …

A Construct Validity Study Of Differentiation Of Self Measures And Their Correlates, Mary Jane Maser

Seton Hall University Dissertations and Theses (ETDs)

.