Mode and Frame Matter: assessing the impact of survey mode and sample frame in choice experiments

Verity Watson* (Corresponding Author), Terry Porteous, Tim Bolt, Mandy Ryan

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)
11 Downloads (Pure)


Background. Choice experiments (CE) are applied in health economics to elicit public preferences and willingness to pay (WTP). CEs are frequently administered as Internet-based surveys. Internet surveys have recognized advantages, but concerns exist about the representativeness of Internet samples, data quality, and the impact on elicited values. Aim. We conducted the first study in health comparing an Internet-based CE survey with the more traditional general population mail survey. We also compared the Internet-based and mail CE surveys with computer-assisted personal interviews (CAPIs), which are commonly used to elicit health state valuations. Methods. Two separate samples were drawn from 2 United Kingdom (UK) volunteer Internet panels (IPs), CAPIs were undertaken with respondents sampled from UK Census Output Areas, and mail surveys were sent to UK households drawn from the postcode address file (PAF). Each mode received more than 1000 respondents. We compared modes and frames using objective measures (response rate, sample representativeness of the UK population, elicited values, theoretical validity, and cost per response) and subjective/self-reported measures (time taken to complete the study, perceived study consequentiality, and stated attribute nonattendance). This study intentionally confounded the survey modes and sample frame by choosing sample frames that are typically used by researchers for each mode. Results. Estimated WTP differs across mode-frame pairs. On most measures, CAPIs dominated. They are more expensive, however. On all measures, except response rates, Internet surveys dominated the mail survey. They were also cheaper. Conclusion. Researchers using IPs should pay attention to response rates and be aware that the quality of IPs differs. Given the importance of perceived consequentiality and attribute attendance in CEs, future research should address their impact across modes and frames.
Original languageEnglish
Pages (from-to)827-841
Number of pages15
JournalMedical Decision Making
Issue number7
Early online date15 Sep 2019
Publication statusPublished - Oct 2019


  • survey mode
  • sample frame
  • choice experiment
  • health care
  • MAIL


Dive into the research topics of 'Mode and Frame Matter: assessing the impact of survey mode and sample frame in choice experiments'. Together they form a unique fingerprint.

Cite this