Social and Behavioral Sciences Commons^™

Many Labs 2: Investigating Variation In Replicability Across Samples And Settings, Richard A. Klein, Michelangelo Vianello, Susan L. O'Donnell, Et Al

Faculty Publications - Psychology Department

We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance (p < .05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion (p < .0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely highpowered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohen’s ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than .20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above .10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and low WEIRDness scores, respectively). Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied.

Acculturation And Internalizing Problems Among Latino Youth: A Meta-Analytic Review, Juventino Hernandez Rodriguez

Graduate Theses and Dissertations

Some studies have found acculturation to be a positive predictor of internalizing problems (i.e., anxiety and depression) in Latino youth (Gonzales et al., 2002), whereas other studies have revealed no relation or a negative relation between acculturation and internalizing problems (Smokowski, Buchanan, & Bacallao, 2009). Narrative reviews of this literature exist (Gonzales et al., 2002; Gonzales et al., 2009) but a quantitative synthesis of the literature has not been conducted. After a systematic literature search that identified 38 studies meeting inclusionary criteria, a meta-analysis was performed to estimate the size and direction of the relation between acculturation and internalizing problems. …

Meta-Analysis Challenges A Textbook Example Of Status Signalling And Demonstrates Publication Bias, Alfredo Sánchez-Tójar, Shinichi Nakagawa, Moisès Sánchez-Fortún, Dominic A. Martin, Sukanya Ramani, Antje Girndt, Veronika Bókony, Bart Kempenaers, András Liker, David F. Westneat, Terry Burke, Julia Schroeder

Biology Faculty Publications

The status signalling hypothesis aims to explain within-species variation in ornamentation by suggesting that some ornaments signal dominance status. Here, we use multilevel meta-analytic models to challenge the textbook example of this hypothesis, the black bib of male house sparrows (Passer domesticus). We conducted a systematic review, and obtained primary data from published and unpublished studies to test whether dominance rank is positively associated with bib size across studies. Contrary to previous studies, the overall effect size (i.e. meta-analytic mean) was small and uncertain. Furthermore, we found several biases in the literature that further question the support available …

Exploring The Neural Mechanisms Of Physics Learning, Jessica E. Bartley

FIU Electronic Theses and Dissertations

This dissertation presents a series of neuroimaging investigations and achievements that strive to deepen and broaden our understanding of human problem solving and physics learning. Neuroscience conceives of dynamic relationships between behavior, experience, and brain structure and function, but how neural changes enable human learning across classroom instruction remains an open question. At the same time, physics is a challenging area of study in which introductory students regularly struggle to achieve success across university instruction. Research and initiatives in neuroeducation promise a new understanding into the interactions between biology and education, including the neural mechanisms of learning and development. These …

Message Framing And Climate Change Communication: A Meta-Analytical Review, Nan Li, Leona Yi-Fan Su

Journal of Applied Communications

This meta-analytic study reviewed experimental studies that examined the effects of message framing on public engagement with climate change. We included 10 studies that used self-reported measures of climate-related attitudes and behaviors, with 26 comparison pairs. The results suggested that message framing generally has a positive effect on individuals’ engagement with climate change and its two sub-categories – behavioral intentions and support for climate policy. More specifically, we found message frames that emphasize the environmental, economic, and moral dimensions of climate change have a small-to-medium size impact on individuals’ engagement with climate change. In contrast, message frames around public health …

Meta-Analysis Of Anger And Persuasion: An Empirical Integration Of Four Models, Nathan Walter, Riva Tukachinsky, Ayellet Pelled, Robin Nabi

Communication Faculty Articles and Research

Despite the increasing use of anger in persuasive messaging, such as political ads and health campaigns, very little is known about when and how anger affects persuasion. Building on theoretical propositions derived from four theoretical models that have addressed the link between anger and persuasion, the current meta-analysis (k = 55, N = 6,805) finds a weak impact of anger on behavior (r = .15, p = .04) and nonsignificant effects on attitudes (r = −.03, p = .30) and intent (r = .06, p = .13). Yet a closer look reveals a more complicated reality, where positive effects are …

Does Transcranial Direct Current Stimulation To The Prefrontal Cortex Affect Social Behavior? A Meta-Analysis, Sarah Beth Bell, Nathan Dewall

Psychology Faculty Publications

This meta-analysis (k = 48, N = 2196) examined the effect of transcranial direct current brain stimulation (tDCS) applied to the prefrontal cortex on a variety of social behaviors, including aggression, overeating, impulsivity, bias, honesty, and risk-taking. tDCS showed an overall significant effect on reducing undesirable behaviors, with an average effect size of d = −0.20. tDCS was most effective at reducing risk-taking behavior, bias, and overeating. tDCS did not affect aggression, impulsivity, or dishonesty. We examined moderators such as brain region of interest, online vs offline stimulation, within- vs between-subjects designs, dose, and duration, but none showed significant …

Executive Functioning In Pediatric Youth: A Meta-Analysis, Draycen D. Decator

College of Science and Health Theses and Dissertations

Executive functioning (EF) represents a set of cognitive skills that are important for daily functioning. EF can be influenced by a number of biopsychosocial factors, many of which are present in the pediatric population (i.e., youth with at least one medical condition). It is important to understand EF in this population as it affects aspects of their physical health (e.g., treatment adherence). Previous meta-analyses have been conducted to examine EF in the pediatric population, and they have generally found deficits in EF compared to healthy peers. However, these previous meta-analyses have only focused on specific medical conditions (e.g., pediatric youth …

A Multi-Level Assessment Of Healthcare Facilities Readiness, Willingness, And Ability To Adopt And Sustain Telehealth Services, Jamie Larson

Theses & Dissertations

Telehealth technologies are becoming more pervasive throughout the healthcare system as a way to provide services to patients that would otherwise have difficulty with access. Currently, little is known about the current state of telehealth use within clinics and hospital in the US. Most studies evaluating telehealth programs are feasibility or small patient outcome studies from one location. Utilizing a hybrid framework combining the levels of complex socio-technical systems with the theory of ready, willing and able. The theory of ready, willing, and able is founded on the basis that these three preconditions need to be met for a change …

An Empirical Demonstration Of The Existence Of Measurement Dependence In The Results Of A Meta-Analysis, William R. Nugent, Sukyung Yoon, Jayme E. Walters

Sociology, Social Work and Anthropology Faculty Publications

Objective: Findings from meta-analytic studies that use standardized mean differences (SMDs) may be overly dependent on the original measures that were used to generate SMDs. This may be particularly true when measures have arbitrary metrics or when measures fail to meet measurement equivalence. We test the hypothesis that in such cases, meta-analytic results may vary significantly— statistically and practically—as a function of the measures used to derive SMDs. Methods: We conducted 5 secondary random-effects meta-analyses of SMDs—each under a different measurement scenario—from a published meta-analysis comparing the efficacy of cognitive–behavioral therapy with that of reminiscence therapy for depression in older …

Social Outcome Following Pediatric Traumatic Brain Injury: A Meta-Analysis, Jonathan James Mietchen

Theses and Dissertations

Objective: Children and adolescents with a history of traumatic brain injury (TBI) are at increased risk for developing social incompetence and impairment in broad psychosocial functioning. The aim of this study was to examine the relationship between history of TBI, social competence, and broad psychosocial functioning using meta-analytic methods. Methods: Studies relating to social outcome following pediatric TBI were searched for using scientific, academic databases. Sixteen studies (N=2,005) met inclusion criteria, and relevant data relating to social functioning was extracted. Meta-analytic methods were used in order to obtain Hedges's g effect size data for mild, moderate, and severe TBI groups. …

Systems Of Quantity Judgment In Various Species: A Meta-Analysis, Tiffany A. Woodard Baker

Dissertations

An abundance of behavioral and neuroimaging literature supports the presence of two cognitive systems for quantity judgments (Agrillo & Bisazza, 2014). In particular, small quantities are thought to be guided by the object-file system, a precise system that uses mental files to map onto real world objects, and large quantities by the approximate number system, an imprecise, estimation system (Dehaene, 1997). Evidence supporting both systems exists in a variety of species including nonhuman primates (Boysen & Hallberg, 2000), birds (Garland, Low, & Burns, 2012), amphibians (Uller, Jaeger, Guidry, & Martin, 2003), and fish (Agrillo, Dadda, Serena, Bisazza, 2009), but support …

Hierarchical Meta-Analysis: A Simulation Study Comparing Classical Random Effects And Fully Bayesian Methods, Nancy R. Andiloro

Dissertations, Theses, and Capstone Projects

Meta-analytic data have a natural hierarchical structure to them, where individuals are nested within studies, and have both within-and between-study variation to model. A random-effects hierarchical linear model is useful to conduct a meta-analysis because it allows one to appropriately parse out the two components of variation that exist within and across studies to determine an observed effect. Empirical Bayes estimation considers the reliability of variance estimates; when the reliability of the effect size estimate for a study is high, substantial weight is placed on that estimate. However, problems with estimation arise when the number of studies and their sample …

A Supramodal Role Of The Basal Ganglia In Memory And Motor Inhibition: Meta-Analytic Evidence, Yuhua Guo, Taylor W. Schmitz, Marieke Mur, Catarina S. Ferreira, Michael C. Anderson

Brain and Mind Institute Researchers' Publications

© 2017 The Authors The ability to stop actions and thoughts is essential for goal-directed behaviour. Neuroimaging research has revealed that stopping actions and thoughts engage similar cortical mechanisms, including the ventro- and dorso-lateral prefrontal cortex. However, whether and how these abilities require similar subcortical mechanisms remains unexplored. Specifically of interest are the basal ganglia, subcortical structures long-known for their motor functions, but less so for their role in cognition. To investigate the potential common mechanisms in the basal ganglia underlying action and thought stopping, we conducted meta-analyses using fMRI data from the Go/No-Go, Stop-signal, and Think/No-Think tasks. All three …

Relative Effectiveness Of Mindfulness And Cognitive Behavioral Interventions For Anxiety Disorders: Meta-Analytic Review, Samina K. Singh, Kevin M. Gorey

Social Work Publications

Increasingly popular mindfulness intervention innovations seem demonstrably effective in alleviating anxiety among people with anxiety disorders. However, the basis of such primary and synthetic evidence has, for the most part, been comparisons with non-active comparison conditions such as waiting lists. The longest-standing and strongest evidence-informed practices in this field have been cognitive behavioral interventions (CBI). This meta-analysis synthesized evidence from nine randomized trials of the relative effectiveness of mindfulness interventions compared to CBIs (i.e., active control groups) in treating anxiety disorders. The sample-weighted synthesis found no statistically or practically significant differences between the two groups on anxiety alleviation: Cohen’s d …

The Efficacy Of Mindfulness Based Interventions For Soldiers And Veterans, Adam Clark

Selected Full Text Dissertations, 2011-

The psychological impacts of war can be devastating for soldiers and veterans. Even for those who have not seen combat, military service can be a stressful experience. Although posttraumatic stress disorder (PTSD) has been identified as the most common form of psychopathology in this population, military personnel are susceptible to a variety of other mental illnesses, including anxiety, depression, and substance use disorders (Prigerson, Maciejewski, & Rosenheck, 2002). Meta-analytic reviews demonstrate the efficacy of exposure-based interventions for soldiers and veterans with symptoms of posttraumatic stress (PTS), but results are inconclusive regarding the efficacy of other treatments for military servicemembers (Kitchiner, …

The Effect Of Laboratory Manipulations Of Negative Affect On Alcohol Craving And Use: A Meta-Analysis, Konrad Bresin, Yara Mekawi, Edelyn Verona

Psychology Faculty Publications

Scientific and lay theories propose that negative affect plays a causal role in problematic alcohol use. Despite this common belief, supporting experimental evidence has been mixed. Thus, the goals of this study were to (a) meta-analytically quantify the degree to which experimentally manipulated negative affect influenced alcohol use and craving in the laboratory, (b) examine whether the size of this effect depended on key manipulation characteristics (i.e., self-relevance of the stressor, timing of the end of the stressor, and strength of negative affect induction) or sample characteristics (i.e., substance use history). Across 41 studies (N = 2,403), we found …

Characterizing Belief Bias In Syllogistic Reasoning: A Hierarchical Bayesian Meta-Analysis Of Roc Data, Dries Trippas, David Kellen, Henrick Singman, Gordon Pennycook, Derek J. Koehler, Jonathan A. Fugelsang, Chad Dubé

Psychology Faculty Publications

The belief-bias effect is one of the most-studied biases in reasoning. A recent study of the phenomenon using the signal detection theory (SDT) model called into question all theoretical accounts of belief bias by demonstrating that belief-based differences in the ability to discriminate between valid and invalid syllogisms may be an artifact stemming from the use of inappropriate linear measurement models such as analysis of variance (Dube et al., Psychological Review, 117(3), 831–863, 2010). The discrepancy between Dube et al.’s, Psychological Review, 117(3), 831–863 (2010) results and the previous three decades of work, together with …

Contact Is A Stronger Predictor Of Attitudes Toward Police Than Race: A State-Of-The-Art Review, Amy Alberton, Kevin M. Gorey

Social Work Publications

Purpose – This scoping review thoroughly scanned research on race, contacts with police and attitudes toward police. An exploratory meta-analysis then assessed the strength of their associations and interaction in Canada and the USA. Key knowledge gaps and specific future research needs, synthetic and primary, were identified. The paper aims to discuss these issues.

Design/methodology/approach – A germinal methodological framework for conducting scoping reviews was used (Arksey and O’Malley, 2005). The authors searched for published or unpublished research over the past 15 years and retrieved 33 eligible surveys, 19 of which were included in a sample-weighted meta-analysis.

Findings – The …

Cultural Adaptations And Therapist Multicultural Competence: Two Meta-Analytic Reviews, Timothy B. Smith, Alberto Soto, Derek Griner, Melanie Domenech Rodriguez, Guillermo Bernal

Faculty Publications

Mental health treatments can be more effective when they align with the culture of the client and when therapists demonstrate multicultural competence. After providing a brief clinical example of cultural adaptations and therapist multicultural competence, we summarize relevant research findings in two meta-analyses. In the meta-analysis on the effectiveness of culturally adapted interventions, the average effect size across 99 studies was d = .50 (.35 after accounting for publication bias). In the second meta-analysis on 15 studies of therapist cultural competence, the results differed by rating source: Client-rated measures of therapist cultural competence correlated strongly (r = .38) with …

Effects Of Parent-Implemented Interventions On Outcomes For Children With Developmental Disabilities: A Meta-Analysis, Tina Taylor, Timothy B. Smith, Byran B. Korth, Barbara Mandleco

Faculty Publications

A large body of literature exists related to parent-implemented interventions for children with disabilities, so it is helpful to synthesize the results of outcome-based interventions for children with developmental disabilities. Specifically, what are the effects of parent-implemented interventions intended to improve children’s (1) social behaviors, (2) life skills/adaptive behavior, and (3) communication skills? Using meta-analytic aggregation of effect sizes across 30 studies with a total of 1,356 participants, this review examined the association between parent-implemented interventions and intended outcomes for young children with developmental disabilities. Across all 30 studies comparing children’s outcomes to control groups, the random-effects-weighted average effect size …

A Meta-Analysis Of Video Based Interventions In Adult Mental Health, Lauretta Kaye Montes

Walden Dissertations and Doctoral Studies

Symptoms of mental illness such as anxiety and depression diminish functioning, cause distress, and create an economic burden to individuals and society. This meta-analysis was designed to evaluate the effectiveness of video based interventions (VBIs) for the treatment of adults in mental health settings. VBIs comprise four different ways of using video in mental health therapy, including video modeling, video exposure, video feedback, and videos used for psychoeducation. Bandura's social learning theory, Beck's cognitive theory, and Dowrick's theory of feedforward learning form the theoretical framework for understanding how VBIs work. The research questions were: (a) what is the range of …