For better or worse? Investigating the validity of best-worst discrete choice experiments in health

Discrete choice experiments (DCEs) are frequently used in health economics to measure preferences for non-market goods. Best worst discrete choice experiment (BWDCE) has been proposed as a variant of the traditional “pick the best” approach. BWDCE, where participants choose the best and worst options, is argued to generate more precise preference estimates because of the additional information collected. However, the validity of the approach relies on two necessary conditions: (i) best and worst decisions provide similar information about preferences, and (ii) asking individuals to answer more than one choice question per task does not reduce data quality. Whether these conditions hold in empirical applications remains under researched. This is the first study to compare participants’ choices across three experimental conditions: (i) BEST choices only, (ii) WORST choices only, and (iii) BEST & WORST choices (BWDCE). We find responses to worst choices are noisier. Implied preferences from the best only and worst only choices are qualitatively different, leading to different WTP values. Responses to BWDCE tasks have lower consistency and respondents are more likely to use simplifying decision heuristics. We urge caution in using BWDCE as an alternative to the traditional "pick the best" DCE.