Digital Commons Network^™

Do You Need A Foot-In-The-Door Or Is A Toe Enough? Scripting Introductions That Induce Tailoring And Increase Participation In Telephone Interviews, Kim Ethridge, Matt Jans, Matthew D. Mcdonough, Sam Vincent, Jamie Dayton, Naomi Freedner, Randal Zuwallack, Josh Duell, Don Allen, Dan Bertuna, Lew Berman, Mark Serafin, Kristin Reichl, Anneke Jansen, Wendi Gilreath

Interviewer Workshop, 2019: Interviewers and Their Effects from a Total Survey Error Perspective

Substantial research and practical experience shows that a telephone interviewer is most successful at gaining cooperation and avoiding refusals when they are free to tailor their introductory pitch to the potential respondent or household informant they reach. However, survey designers are often uncomfortable allowing interviewers to work “off-script,” and instruct interviewers to read introductory text verbatim. Further, some interviewers report being more comfortable with a script than without one. To bridge this gap between research and practice we asked, “Can we create a scripted introduction that engages the potential respondent, gets a foot-in-the-door, and facilitates interviewer tailoring?” This paper reports …

Analysing The Influence Of Non-Observable And Observable Interviewer Characteristics On Measurement Error: Evidence From Zambia, P. Linh Nguyen

Interviewer Workshop, 2019: Interviewers and Their Effects from a Total Survey Error Perspective

In Sub-Saharan Africa, where only one in five people uses the Internet and connectivity issues restrict the possibility for phone surveys in rural areas, interviewer-administered face-to-face (F2F) surveys are and will remain the principal data collection tool in the foreseeable future. Yet questions remain as to what extent previous findings on interviewer-administered surveys from Western countries may apply to a different cultural and geographical context. In this light, the objective of this study is to investigate the influence of certain observable interviewer characteristics (such as gender, age) and non-observable characteristics (such as education, attitudes) on interviewer variance on a subset …

Examining Interviewer Effects On The Agricultural Labor Survey: A Mixed-Methods Approach, David Biagas, Emilola Abayomi, Joseph Rodhouse, Heather Ridolfo

Interviewer Workshop, 2019: Interviewers and Their Effects from a Total Survey Error Perspective

Interviewer effects remain a pervasive problem in survey research. Utilizing a mixed-methods approach, this study explores the effects that interviewers have on the reporting of agricultural workers on the USDA’s Agricultural Labor Survey. The Agricultural Labor Survey is administered by the USDA’s National Agricultural Statistics Service on a biannual basis, with each data collection period collecting information for two quarters (e.g. April and January of 2018). While the majority of data is collected via computer-assisted telephone interviewing, a sizeable proportion is completed online or via mail. Using quantitative and qualitative methods, this study explores the patterns of bias introduced by …

Did The Respondent Really Mean That? How The Behaviors Of Cati Interviewers And Data Editors Impact Measurement And Processing Errors In Establishment Surveys, Joseph Rodhouse, Heather Ridolfo, Emilola Abayomi, David Biagas

Interviewer Workshop, 2019: Interviewers and Their Effects from a Total Survey Error Perspective

It is well documented that interviewers can have profound effects on the survey data collection process. This research looks to build on that knowledge by examining the relationship between CATI interviewers and data editors and how the recording of answers, and editing of the survey answers recorded, contribute to total survey error (TSE). Specifically, we are interested in comparing answers recorded by CATI interviewers and the final response codes after the editing stage. Since the interviewing stage and the editing (or processing) stage are often two distinct phases of the data collection process where the interviewer and data editor work …

Implementing A Case Ownership Model With Telephone Interviewers, Jamie Wescott

Interviewer Workshop, 2019: Interviewers and Their Effects from a Total Survey Error Perspective

In an interviewer-administered survey, interviewers themselves can have a substantial impact on survey quality. In their review of the literature, West and Blom (2017) described the innumerable articles dedicated to describing this effect. A subset of these have provided support for the linkage between interviewer variation, such as differences in experience level and contacting approach, and variation in unit noncontact and nonresponse rates; these include Purdon, Campanelli, and Sturgis (1999), Groves and Couper (1998), and Blom (2012). In telephone studies, automated call scheduling systems help to mitigate these effects by using algorithms to ensure that cases receive an appropriate number …

Developing A Quality Control Protocol For Evaluation Of Recorded Interviews, Margaret Hudson, Lisa Holland, Lisa Lewandowski-Romps

Interviewer Workshop, 2019: Interviewers and Their Effects from a Total Survey Error Perspective

This presentation will describe the process used at the University of Michigan Survey Research Center for evaluating interviewer performance in survey administration. Within the Survey Research Operations unit, we use an online system for evaluating the interviewer-respondent interaction using recorded interviews. We will present our framework for measuring how well interviewers adhere to General Interviewing Techniques (GIT) - the guidelines in which they were trained. The presentation will describe the question-level and session-level measurement criteria employed, in addition to the selection protocols and the integration of paradata into the selection process. The presentation will include analysis of some evaluation data, …

Audio Recordings In Face-To-Face Interviews As A Means To Detect Undesirable Interviewer Behavior, Birgit Jesske

Interviewer Workshop, 2019: Interviewers and Their Effects from a Total Survey Error Perspective

Undesirable interviewer behavior (UIB) could be one source for data errors and measurement effects in the setting of standardized interviewing techniques. Survey organizations have to ensure that errors and effects are minimized by validating their data collection processes during the entire survey period.

Monitoring is one method of validation which has been well established for telephone surveys from their very beginning. Moreover, it is one of the advantage of telephone interviews compared to face-to-face interviews. In most survey organizations it includes listening to interviews at the time they are being carried out by either supervisors or clients resp. scientists.

For …

Behavior Change Techniques For Reducing Interviewer Contributions To Total Survey Error, Brad Edwards

Interviewer Workshop, 2019: Interviewers and Their Effects from a Total Survey Error Perspective

In the total survey error paradigm, nonsampling errors can be difficult to quantify, especially errors that occur in the data collection phase of face-to-face surveys. Field interviewers play “… dual roles as recruiters and data collectors…” (West et al, 2018), and are therefore potential contributors to both nonresponse error and measurement error. Recent advances in technology, paradata, and performance dashboards offer an opportunity to observe interviewer effects almost at the source in real time, and to intervene quickly to curtail them. Edwards, Maitland, and Connor (2017) report on an experimental program using rapid feedback of CARI coding results (within 72 …