Open Access. Powered by Scholars. Published by Universities.®

Quantitative, Qualitative, Comparative, and Historical Methodologies Commons

Open Access. Powered by Scholars. Published by Universities.®

3,148 Full-Text Articles 4,192 Authors 3,030,704 Downloads 139 Institutions

All Articles in Quantitative, Qualitative, Comparative, and Historical Methodologies

Faceted Search

3,148 full-text articles. Page 4 of 116.

Comparison Of Different Approaches To Evaluate And Explain Interviewer Effects, Geert Loosveldt, Celine Wuyts 2019 KU Leuven

Comparison Of Different Approaches To Evaluate And Explain Interviewer Effects, Geert Loosveldt, Celine Wuyts

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

Within survey methodology it is common knowledge that interviewers in face-to-face or telephone interviews can have undesirable effects on the obtained answers. These effects can be created in an active way by, for example, asking suggestive questions or they can be obtained in a passive way as a consequence of certain interviewer characteristics eliciting socially desirable answers. These active and passive effects may differ from interviewer to interviewer. These differences between interviewers in systematic effects create additional variance in the data. The proportion of variance in a (substantive) variable that can be explained by the interviewers is the ‘so called ...


Designing Studies For Comparing Interviewer Variance Components In Two Groups Of Survey Interviewers, Brady T. West 2019 University of Michigan, Ann Arbor

Designing Studies For Comparing Interviewer Variance Components In Two Groups Of Survey Interviewers, Brady T. West

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

Methodological studies of interviewer effects often seek to identify factors that influence the magnitude of interviewer variance for particular survey questions. There is a long history of work in this area, and results from studies like this have informed current interviewing practice. Unfortunately, many studies of this type suffer from one or more of the following limitations in terms of their designs: 1) a failure to randomly assign interviewers to the treatments being compared; 2) a failure to formally test for differences in the variance components between the two groups; and 3) insufficient statistical power for comparison of the variance ...


How To Conduct Effective Interviewer Training: A Meta-Analysis, Jessica Daikeler 2019 GESIS – Leibniz Institute for the Social Sciences and University of Mannheim

How To Conduct Effective Interviewer Training: A Meta-Analysis, Jessica Daikeler

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

Interviewer training can improve the performance of interviewers and thus also the quality of survey data. However, the question of how effective interviewer training is for improving data quality and more importantly, which determinates drive its success, remain unanswered. This research uses meta-analytical methods to evaluate both the improvements in data quality due to interviewer training and the effectivity of training modules with respect interviewer performance. We consider various aspects of data quality, namely unit nonresponse, item nonresponse, probing behavior, administration, reading, and recording. Based on more than sixty experimental studies, we find that comprehensive interviewer training improves unit- and ...


Analysing The Influence Of Non-Observable And Observable Interviewer Characteristics On Measurement Error: Evidence From Zambia, P. Linh Nguyen 2019 , University of Essex/University of Mannheim

Analysing The Influence Of Non-Observable And Observable Interviewer Characteristics On Measurement Error: Evidence From Zambia, P. Linh Nguyen

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

In Sub-Saharan Africa, where only one in five people uses the Internet and connectivity issues restrict the possibility for phone surveys in rural areas, interviewer-administered face-to-face (F2F) surveys are and will remain the principal data collection tool in the foreseeable future. Yet questions remain as to what extent previous findings on interviewer-administered surveys from Western countries may apply to a different cultural and geographical context. In this light, the objective of this study is to investigate the influence of certain observable interviewer characteristics (such as gender, age) and non-observable characteristics (such as education, attitudes) on interviewer variance on a subset ...


Unintended Interviewer Bias In A Community-Based Participatory Research Randomized Control Trial Among American Indian Youth, Patrick Habecker, Jerreed Ivanich 2019 University of Nebraska-Lincoln

Unintended Interviewer Bias In A Community-Based Participatory Research Randomized Control Trial Among American Indian Youth, Patrick Habecker, Jerreed Ivanich

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

Community-based participatory research (CBPR) projects often employ members of the host partner community to engage and assist with research projects. However, CBPR may also introduce bias to survey statistics when community partners work as interviewers for projects within their own communities. Here, the advantage of employing interviewers from the local community and region may lead to unintended bias when participants and interviewers know each other outside of the research project. In situations where a preexisting social relationship exists, there is a greater possibility of social desirability bias. This may be particularly true for sensitive issues where they may not wish ...


Examining Interviewer Effects On The Agricultural Labor Survey: A Mixed-Methods Approach, David Biagas, Emilola Abayomi, Joseph Rodhouse, Heather Ridolfo 2019 Bureau of Labor Statistics

Examining Interviewer Effects On The Agricultural Labor Survey: A Mixed-Methods Approach, David Biagas, Emilola Abayomi, Joseph Rodhouse, Heather Ridolfo

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

Interviewer effects remain a pervasive problem in survey research. Utilizing a mixed-methods approach, this study explores the effects that interviewers have on the reporting of agricultural workers on the USDA’s Agricultural Labor Survey. The Agricultural Labor Survey is administered by the USDA’s National Agricultural Statistics Service on a biannual basis, with each data collection period collecting information for two quarters (e.g. April and January of 2018). While the majority of data is collected via computer-assisted telephone interviewing, a sizeable proportion is completed online or via mail. Using quantitative and qualitative methods, this study explores the patterns of ...


Did The Respondent Really Mean That? How The Behaviors Of Cati Interviewers And Data Editors Impact Measurement And Processing Errors In Establishment Surveys, Joseph Rodhouse, Heather Ridolfo, Emilola Abayomi, David Biagas 2019 National Institute of Statistical Services and National Agricultural Statistics Service

Did The Respondent Really Mean That? How The Behaviors Of Cati Interviewers And Data Editors Impact Measurement And Processing Errors In Establishment Surveys, Joseph Rodhouse, Heather Ridolfo, Emilola Abayomi, David Biagas

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

It is well documented that interviewers can have profound effects on the survey data collection process. This research looks to build on that knowledge by examining the relationship between CATI interviewers and data editors and how the recording of answers, and editing of the survey answers recorded, contribute to total survey error (TSE). Specifically, we are interested in comparing answers recorded by CATI interviewers and the final response codes after the editing stage. Since the interviewing stage and the editing (or processing) stage are often two distinct phases of the data collection process where the interviewer and data editor work ...


Implementing A Case Ownership Model With Telephone Interviewers, Jamie Wescott 2019 RTI International

Implementing A Case Ownership Model With Telephone Interviewers, Jamie Wescott

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

In an interviewer-administered survey, interviewers themselves can have a substantial impact on survey quality. In their review of the literature, West and Blom (2017) described the innumerable articles dedicated to describing this effect. A subset of these have provided support for the linkage between interviewer variation, such as differences in experience level and contacting approach, and variation in unit noncontact and nonresponse rates; these include Purdon, Campanelli, and Sturgis (1999), Groves and Couper (1998), and Blom (2012). In telephone studies, automated call scheduling systems help to mitigate these effects by using algorithms to ensure that cases receive an appropriate number ...


Developing A Quality Control Protocol For Evaluation Of Recorded Interviews, Margaret Hudson, Lisa Holland, Lisa Lewandowski-Romps 2019 Survey Research Center, University of Michigan

Developing A Quality Control Protocol For Evaluation Of Recorded Interviews, Margaret Hudson, Lisa Holland, Lisa Lewandowski-Romps

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

This presentation will describe the process used at the University of Michigan Survey Research Center for evaluating interviewer performance in survey administration. Within the Survey Research Operations unit, we use an online system for evaluating the interviewer-respondent interaction using recorded interviews. We will present our framework for measuring how well interviewers adhere to General Interviewing Techniques (GIT) - the guidelines in which they were trained. The presentation will describe the question-level and session-level measurement criteria employed, in addition to the selection protocols and the integration of paradata into the selection process. The presentation will include analysis of some evaluation data, with ...


Audio Recordings In Face-To-Face Interviews As A Means To Detect Undesirable Interviewer Behavior, Birgit Jesske 2019 infas Institute for Applied Social Sciences

Audio Recordings In Face-To-Face Interviews As A Means To Detect Undesirable Interviewer Behavior, Birgit Jesske

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

Undesirable interviewer behavior (UIB) could be one source for data errors and measurement effects in the setting of standardized interviewing techniques. Survey organizations have to ensure that errors and effects are minimized by validating their data collection processes during the entire survey period.

Monitoring is one method of validation which has been well established for telephone surveys from their very beginning. Moreover, it is one of the advantage of telephone interviews compared to face-to-face interviews. In most survey organizations it includes listening to interviews at the time they are being carried out by either supervisors or clients resp. scientists.

For ...


Modelling Group-Specific Interviewer Effects On Nonresponse Using Separate Coding For Random Slopes In Multilevel Models, Jessica M. E. Herzing, Annelies G. Blom, Bart Meuleman 2019 LINES/FORS, University of Lausanne, Switzerland

Modelling Group-Specific Interviewer Effects On Nonresponse Using Separate Coding For Random Slopes In Multilevel Models, Jessica M. E. Herzing, Annelies G. Blom, Bart Meuleman

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

To enhance response among underrepresented groups and hence, to increase response rates and to decrease potential nonresponse bias survey practitioners often use interviewers in population surveys (Heerwegh, 2009). While interviewers tend to increase overall response rates in surveys (see Heerwegh, 2009), research on the determinants of nonresponse have also identified human interviewers as one reason for variations in response rates (see for examples Couper & Groves, 1992; Durrant, Groves, Staetsky, & Steele, 2010; Durrant & Steele, 2009; Hox & de Leeuw, 2002; Loosveldt & Beullens, 2014; West & Blom, 2016). In addition, research on interviewer effects indicates that interviewers introduce nonresponse bias, if interviewers systematically differ in their success in obtaining response from specific respondent groups (see West, Kreuter, & Jaenichen, 2013; West & Olson, 2010). Therefore, interviewers might be a source of selective nonresponse in surveys.

Interviewers might also differentially contribute to selective nonresponse in surveys and hence, potential nonresponse bias, when interviewer effects are correlated with characteristics of the approached sample units (for an example see Loosveldt & Beullens, 2014). Multilevel models including dummies in the random part of the model to distinguish between respondent groups are commonly used to investigate whether interviewer effects on nonresponse differ across specific respondent groups (see Loosveldt & Beullens, 2014). When dummy coding, which is also referred to as contrast coding (Jones, 2013), are included as random components in multilevel models for interviewers effects, the obtained variance estimates indicate to what extent the contrast between respondent groups varies across interviewers. Yet, such parameterization does not directly yield insight on the size of interviewer effects for specific respondent groups.

Surveys with large imbalances among respondent groups gain from an investigation of the variation of interviewer effect sizes on nonresponse, as one gains insights on whether the interviewer effect size is the same for specific respondent groups. The importance of the interviewer effect size for specific groups of respondents lies in its prediction of the effectiveness of interviewer-related fieldwork strategies (for examples on liking, matching, or prioritizing respondents with interviewers see Durrant et al., 2010; Peytchev, Riley, Rosen, Murphy, & Lindblad, 2010; Pickery & Loosveldt, 2002, 2004) and thus, a effective mitigation of potential nonresponse bias. Consequently, understanding group-specific interviewer effect sizes can aide the efficiency of respondent recruitment, because we then understand why some interviewer-related fieldwork strategies have great impact on some respondent group’s participation while other strategies have little effect.

To obtain information on differences in interviewer effect size, we propose to use an alternative coding strategy, so-called separate coding in multilevel models with random slopes (for examples see Jones, 2013; Verbeke & Molenberghs, 2000, ch. 12.1). In case of separate coding, every variable represents a direct estimate of the interviewer effects for specific ...


Explaining Interviewer Effects On Survey Unit Nonresponse: A Cross-Survey Analysis, Daniela Ackermann-Piek, Annelies G. Blom, Julie M. Korbmacher, Ulrich Krieger 2019 GESIS - Leibniz Institute for the Social Sciences

Explaining Interviewer Effects On Survey Unit Nonresponse: A Cross-Survey Analysis, Daniela Ackermann-Piek, Annelies G. Blom, Julie M. Korbmacher, Ulrich Krieger

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

In interviewer-administered surveys, interviewers are involved in nearly all steps of the survey implementation. However, besides many positive aspects of interviewers’ involvement, they are – intentionally or unintentionally – a potential source of survey errors. In recent decades, a large body of literature has accumulated about measuring and explaining interviewer effects on survey unit nonresponse. Recently, West and Blom (2017) have published a research synthesis on factors explaining interviewer effects on various sources of survey error, including survey unit nonresponse. They find that previous research reports great variability across surveys in the significance and even direction of predictors of interviewer effects on ...


General Interviewing Techniques: Developing Evidence-Based Practices, Steve Coombs, Margaret Hudson, Lisa Holland, Nora Cate Schaeffer, Jennifer Dykema 2019 University of Wisconsin Survey Center

General Interviewing Techniques: Developing Evidence-Based Practices, Steve Coombs, Margaret Hudson, Lisa Holland, Nora Cate Schaeffer, Jennifer Dykema

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

This poster is a hands-on demonstration of the in-progress General Interviewer Techniques (GIT) materials described by Schaeffer, Dykema, Coombs, Schultz, Holland, and Hudson. Participants will be able to view and listen to the lesson materials, delivered via an online interface, and talk to the GIT developers.


Effects Of Innovative Motivational Strategies And New Staffing Model On Interviewer Attrition: A Data Collection Year In Review, Theresa Camelo, Maureen O’Brien 2019 Survey Research Center, University of Michigan

Effects Of Innovative Motivational Strategies And New Staffing Model On Interviewer Attrition: A Data Collection Year In Review, Theresa Camelo, Maureen O’Brien

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

Due to high staff attrition and its negative effects on data collection and project cost, a large national study implemented motivational strategies and a new staffing model for the current data collection year. Motivational strategies included retention bonuses, organization gear, and other personalized recognitions. The new staffing model included both a change in weekly hour requirements as well as the number of interviewers staffed in each area. New staff, committed to 20 hours per week, were added to approximately half of the sampling areas with already existing 30 hour per week staff. Two Interviewers were now working a single area ...


Exploring The Mind Of The Interviewer: Findings From Research With Interviewers To Improve The Survey Process, Robin Kaplan, Erica Yu 2019 Bureau of Labor Statistics

Exploring The Mind Of The Interviewer: Findings From Research With Interviewers To Improve The Survey Process, Robin Kaplan, Erica Yu

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

The interviewers’ task in the data collection process is a complex one, with many judgments and decisions being made from moment to moment as they ask questions to get answers from respondents (Japec, 2008). Many survey organizations train their interviewers to use standardized language and read questions verbatim. However, in practice, interviewers may need to use a conversational approach and probe respondents to get the answers needed. This research explores the process by which interviewers make such decisions in real-time by conducting research with interviewers about their experiences collecting data. Using a cognitive interview approach, we asked interviewers about multiple ...


How Customization Affects Survey Interaction, Antje Rosebrock, Malte Schierholz 2019 University of Mannheim

How Customization Affects Survey Interaction, Antje Rosebrock, Malte Schierholz

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

One common trend in the world of survey data collection is the increasing use of new technological developments which can change the nature of the survey interview. A fairly recent trend is the use of machine-learning techniques to customize questions for respondents. This has the potential to create an individualized experience for the respondent and to improve data quality. Nevertheless, little is known so far of how customization affects the interaction in the survey interview.

We introduce a tool developed by Schierholz et al. (2018) to code respondents’ occupation categories during the survey. The tool uses supervised learning algorithms to ...


Can Paradata Predict Interviewer Effects?, Sharan Sharma 2019 University of Michigan

Can Paradata Predict Interviewer Effects?, Sharan Sharma

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

Consideration of interviewer effects (interviewer measurement error variance) in active quality control does not seem widespread despite its known effect on reducing precision of survey estimates. One major obstacle is that interviewer effect estimates computed on partial data (as a survey is in progress) can be very unstable. We address this issue by exploring the use of paradata (keystrokes and time stamps generated during the computer-assisted interviewing process) as proxies of interviewer effects with a focus on large-scale repeated cross-section or panel surveys.

We first estimate interviewer effects for each item in our analysis by using multilevel models that include ...


Humans Vs. Machines: Comparing Coding Of Interviewer Question-Asking Behaviors Using Recurrent Neural Networks To Human Coders, Jerry Timbrook, Adam Eck 2019 University of Nebraska-Lincoln, Department of Sociology

Humans Vs. Machines: Comparing Coding Of Interviewer Question-Asking Behaviors Using Recurrent Neural Networks To Human Coders, Jerry Timbrook, Adam Eck

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

Standardized survey interviewing techniques are intended to reduce interviewers’ effects on survey data. A common method to assess whether or not interviewers read survey questions exactly as worded is behavior coding. However, manually behavior coding an entire survey is expensive and time-consuming. Machine learning techniques such as Recurrent Neural Networks (RNNs) may offer a way to partially automate this process, saving time and money. RNNs learn to categorize sequential data (e.g., conversational speech) based on patterns learned from previously categorized examples. Yet the feasibility of an automated RNN-based behavior coding approach and how accurately this approach codes behaviors compared ...


Do You Need A Foot-In-The-Door Or Is A Toe Enough? Scripting Introductions That Induce Tailoring And Increase Participation In Telephone Interviews, Kim Ethridge, Matt Jans, Matthew D. McDonough, Sam Vincent, Jamie Dayton, Naomi Freedner, Randal ZuWallack, Josh Duell, Don Allen, Dan Bertuna, Lew Berman, Mark Serafin, Kristin Reichl, Anneke Jansen, Wendi Gilreath 2019 ICF

Do You Need A Foot-In-The-Door Or Is A Toe Enough? Scripting Introductions That Induce Tailoring And Increase Participation In Telephone Interviews, Kim Ethridge, Matt Jans, Matthew D. Mcdonough, Sam Vincent, Jamie Dayton, Naomi Freedner, Randal Zuwallack, Josh Duell, Don Allen, Dan Bertuna, Lew Berman, Mark Serafin, Kristin Reichl, Anneke Jansen, Wendi Gilreath

2019 Workshop: Interviewers and Their Effects from a Total Survey Error Perspective

Substantial research and practical experience shows that a telephone interviewer is most successful at gaining cooperation and avoiding refusals when they are free to tailor their introductory pitch to the potential respondent or household informant they reach. However, survey designers are often uncomfortable allowing interviewers to work “off-script,” and instruct interviewers to read introductory text verbatim. Further, some interviewers report being more comfortable with a script than without one. To bridge this gap between research and practice we asked, “Can we create a scripted introduction that engages the potential respondent, gets a foot-in-the-door, and facilitates interviewer tailoring?” This paper reports ...


“Build A Bridge So You Can Cross It:” A Photo-Elicitation Study Of Health And Wellness Among Homeless And Marginally Housed Veterans, Keri L. Rodriguez, Lauren M. Broyles, Michael A. Mitchell, Melissa E. Wieland, Gala True, Adam J. Gordon 2019 Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA, USA; Division of General Internal Medicine, Department of Medicine, School of Medicine, University of Pittsburgh, PA, USA

“Build A Bridge So You Can Cross It:” A Photo-Elicitation Study Of Health And Wellness Among Homeless And Marginally Housed Veterans, Keri L. Rodriguez, Lauren M. Broyles, Michael A. Mitchell, Melissa E. Wieland, Gala True, Adam J. Gordon

The Qualitative Report

As part of a photo-elicitation interview study, we aimed to describe homeless and marginally housed Veterans’ experiences with health and wellness, health decisions, and health-related behaviors. Twenty Veterans receiving Veterans Affairs Homeless Patient-Aligned Care Team care took photographs depicting health and wellness, then used their photographs to discuss the same topics in 30-60 minute audio-recorded, semi-structured photo-elicitation interviews. Transcripts were analyzed using template analysis. Veterans described eight dimensions related to their health and wellness; physical, social, and environmental were most commonly discussed, followed by emotional, intellectual, spiritual, occupational, and financial wellness. Photographs contained literal and metaphorical depictions that were positively-oriented ...


Digital Commons powered by bepress