The International Total Survey Error Workshop (ITSEW) was held on June 10-12 at the University of Bergamo, Italy brought together survey researchers, practitioners, and methodologists (especially in national statistics offices) to foster an exchange of ideas and preliminary research findings toward a better understanding of total survey error. This workshop has been held annually since 2005 and typically gathers 50-60 attendees.
The goals of the workshop include:
- Reviewing progress on important TSE problems
- Defining current data quality problems in detail, and articulating a research agenda to address them
- Forming research collaborations to carry out needed research
- Identifying emerging research needs at an early stage
- Investigating the quality of alternative data sources.
The topic of the 2019 edition of ITSEW was "Integration of surveys and alternative data sources". Before the start of the workshop on Monday, June 10, there was a short primer on Total Survey Error. This primer reviewed the literature on Total Survey Error and a discussion of recommended best practices.
The 2019 Workshop Program (with links to slides)
MONDAY, 10 June 2019
Primer on TSE: Total Survey Error; Roots and Evolution - Lars Lyberg, Stockholm University
SESSION 1: Innovations at National Statistical Institutes
Chair: Martin Beaulieu, Statistics Canada
Effects of Istat CDC (Centralised Data Collection) approach on the reduction of the Total Survey Error - Loredana De Gaetano and Pasquale Papa*, Italian National Statistical Institute.
Placing administrative data at the heart of the UK population statistics system; how the total survey error framework is helping to inform the use and design of integrated data solutions - Louisa Blackwell and Nicola Rogers*, Office for National Statistics.
Adaptive survey design at Statistics Netherlands - Kees van Berkel*, Statistics Netherlands.
Total Error and variability measures with integrated disclosure limitation for the quarterly workforce indicators - Kevin L. McKinney(1)*, Andrew S. Green(2), Lars Vilhuber(3), and John M. Abowd(1) (1) U.S. Census Bureau, (2) OECD, (3) Cornell University.
SESSION 2: Integrating probability and non probability samples
Chair: Anders Wallgren, Formerly of the Department of Research and Development at Statistics Sweden
The AmeriSpeak® experience: methods of using a national probability sample panel for studies that combine probability and nonprobability samples - Vicki Pineau*, Ed Mulrow, Nada Ganesh, Michael Yang, and J. Michael Dennis NORC at the University of Chicago.
Data integration of national surveys and nonprobability samples: balancing enhanced analytic capacity within representational constraints - Steven B. Cohen*, RTI International.
Integrating different data sources for total estimation with unknown population size - Zhaoce Liu* and Lynne Stokes, Southern Methodist University.
TUESDAY, 11 June 2019
Keynote Address: Total Survey Error for longitudinal surveys - Peter Lynn, University of Essex
Chair: Paul Biemer, RTI International
SESSION 3: TSE in multi-sources and mixed-mode approaches
Chair: Silvia Biffignandi, University of Bergamo
Evaluating mode effects for an in-person panel survey that transitioned to a mixed web mode - Paul Biemer*, RTI International.
A new Total Error Framework for multi-source processes - Fabiana Rocci, Roberta Varriale*, and Orietta Luzi, Italian National Statistical Institute.
Building a Quality Indicators Framework in a multi-source environment - Martin Beaulieu*, Ryan Chepita, and Susie Fortier, Statistics Canada.
The Total Survey Error – A time series and system approach - Anders Wallgren* and Britt Wallgren, Formerly of the Department of Research and Development at Statistics Sweden
SESSION 4: Data Quality
Chair: Alan F. Karr, RTI International
Assessing interviewer effects in the BRFSS - Ting Yan(1), Antonia Warren(1), Carol Pierannunzi(2), Doug Williams(1), Jennifer Crafts(1), and Sonya Gamble(2), (1) Westat, (2) Centers for Disease Control.
Keep the baby, throw out the bath water: The promises of real-time data quality evaluations Second generation at school - Steven Snell and Carol Haney, Qualtrics
Second generation at school - Daniela Cocchi and Francesco Giovinazzi, University of Bologna
SESSION 5: TSE evaluation
Chair: Oliver Lipps, FORS Lausanne
Considering digital privacy attitude measures in the context of TSE: Results from an international workshop - David L. Vannette(1)*, Frauke Kreuter(2), and Julia Lane(3); (1) Facebook, (2) University of Maryland & IAB, (3) New York University.
Social media data for social indicators: assessing the quality through case studies - Silvia Biffignandi(1), Annamaria Bianchi(1)*, and Camilla Salvatore(2); (1) University of Bergamo, (2) University of Milano Bicocca
Applications of R Shiny to explore, evaluate and improve Total Survey Quality - Xiaodan Lyu*, Heike Hofmann, Emily Berg, and Jie Li, Iowa State
WEDNESDAY, 12 June 2019
SESSION 6: Nonresponse modeling
Chair: Annamaria Bianchi, University of Bergamo
Predictive modeling as an alternative to (re-)weighting - Alan F. Karr*, RTI International
In an era of declining response rates, a note on the importance of estimation procedures - Martin Klein, Joanna Fane Lineback*, and Joseph L. Schafer, U.S. Census Bureau.
The effect of measurement error on clustering algorithms: a sensitivity analysis - Paulina Pankowska(1)*, Daniel Oberski(2), and Dimitris Pavlopoulos(1); (1) Vrije Universiteit Amsterdam, (2) Utrecht University.
SESSION 7: Undercoverage and nonresponse
Chair: David L. Vannette, Facebook
Undercoverage and nonresponse as sources of representativeness bias in nonprobability online panels. The Italian case - Chiara Respi* and Emanuela Sala, University of Milano Bicocca
“Can’t live with ‘em, can’t live without ‘em” Assumptions – the necessary evil - Abby Morgan*, Rebecca Green, and Patrick Graham, Statistics New Zealand
Utilising interviewer observations on housing unit characteristics in the Hungarian LFS nonresponse analysis: a research plan - Linda Mohay and Ferenc Mújdricza*, Hungarian Central Statistical Office
SESSION 8: Administrative data and errors
Chair: Roberta Varriale, Italian National Statistical Institute
Integrating administrative and survey agricultural data with Statistical Matching - Riccardo D’Alberto* and Meri Raggi, University of Bologna.
Minimizing sampling error: Preliminary findings of sampling design simulations using survey and administrative data - Gró Einarsdóttir* and Anton Örn Karlsson, Statistics Iceland.
Are screener questions measures of individual-level general attentiveness to surveys? Evidence from an Italian electoral panel study - Riccardo Ladini* and Cristiano Vezzoni, University of Milan.
SESSION 9: Nonresponse bias assessment
Chair: Vicki Pineau, NORC at the University of Chicago
Does adding a survey language pay off? A method for analysing the potential for reducing representation bias - Micheal Ochsner(1,2) and Oliver Lipps(1)*; (1) FORS Lausanne, (2) ETH Zurich.
Assessing nonresponse bias by permitting individuals to opt out of a survey - Taylor Lewis*, RTI International.
Strategies to minimize non-response bias - Nils Galberg Enoksen*, Bo Bilde, and Monika Klingsbjerg-Besrechel, Danmarks Statistik.
SESSION 10: Non-survey data sources and errors
Chair: Kevin L. McKinney, U.S. Census Bureau
Measuring statistical uncertainty in admin-based population estimates - Louisa Blackwell(1)*, Katy Stokes(1), and Peter W. Smith(2); (1) Office for National Statistics, (2) University of Southampton.
Estimates of accuracy of statistical registers based on Anticipated Variance - Giorgio Alleva(1), Piero Demetrio Falorsi(2), Francesca Petrarca(2)*, and Paolo Righi(2); (1) Sapienza University of Rome, (2) Italian National Statistical Institute.
Correcting survey measurement error using road sensor data - Jonas Klingwort(1,2)*, Bart Buelens(3), Joep Burger(1), and Rainer Schnell(2); (1) Statistics Netherlands, (2) University of Duisburg-Essen, (3) Flemish Institute for Technological Research.