The hazards of extracting hazard ratios from survival curves for meta-analyses in observational studies

Research output: Contribution to conferencePaper

Abstract

Background: Meta-analyses of time-to-event data require the log hazard ratio (HR) and its variance but these data are frequently not reported. Methods exist to extract this information from survival curves in reports of randomised controlled trials. However, it is unclear whether these methods give unbiased estimates when applied to non-randomised studies (NRS).
Objectives: To compare estimates of log HRs extracted from published survival curves to reported adjusted log HRs in NRS.
Methods: Data were extracted from NRS retrieved for a systematic review of surgical interventions for localised renal cell carcinoma. Survival probabilities were extracted from survival curve graphs using digital imaging software independently by two reviewers. Estimates of log HRs were then calculated from extracted data using methods outlined in Parmar (1998) and Williamson (2002). These estimated log HRs were compared to reported log HRs derived from models that attempted to adjust for potential selection bias.
Results: There were five studies that reported both a survival curve and adjusted HR. Of these only one estimated log HR was close to the adjusted log HR. Four NRS had estimated log HR that were different to the reported adjusted log HR and the direction difference was inconsistent (differences on the log HR scale -0.16, 0.26, 0.27 and 0.65).
Conclusions: In this small case study, it was empirically evident that using these methods to extract log HR from survival curves in NRS resulted in biased estimates when compared to adjusted log HR from more suitable models. In this example the potentially biased estimates of HRs was one of several reasons against a formal meta-analysis using such data. In a wider context, if reports of NRS do not include an appropriately estimated log HR (and/or information to derive its variance), reviewers should beware of the pitfalls of extracting this information from survival curves.

References
Parmar MK, Torri V & Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Statistics in Medicine 1998; 17:2815-34
Williamson P, Tudur Smith C, Hutton JL & Marson AG. Aggregate data meta-analysis with time-to-event outcomes. Statistics in Medicine 2002; 21:3337-51
Original languageEnglish
Publication statusPublished - Oct 2011
EventCochrane Colloquium 2011 - Madrid, Spain
Duration: 18 Oct 201122 Oct 2011

Conference

ConferenceCochrane Colloquium 2011
CountrySpain
CityMadrid
Period18/10/1122/10/11

Fingerprint

Observational Studies
Meta-Analysis
Medicine
Selection Bias
Renal Cell Carcinoma
Software
Randomized Controlled Trials

Cite this

The hazards of extracting hazard ratios from survival curves for meta-analyses in observational studies. / Imamura, Mari; MacLennan, Steven; Dahm, P; Nueberger, Molly; Omar, Muhammad Imran; MacLennan, Graeme Stewart.

2011. Paper presented at Cochrane Colloquium 2011, Madrid, Spain.

Research output: Contribution to conferencePaper

Imamura, M, MacLennan, S, Dahm, P, Nueberger, M, Omar, MI & MacLennan, GS 2011, 'The hazards of extracting hazard ratios from survival curves for meta-analyses in observational studies' Paper presented at Cochrane Colloquium 2011, Madrid, Spain, 18/10/11 - 22/10/11, .
@conference{8733810071fd447696bc010e02a4681c,
title = "The hazards of extracting hazard ratios from survival curves for meta-analyses in observational studies",
abstract = "Background: Meta-analyses of time-to-event data require the log hazard ratio (HR) and its variance but these data are frequently not reported. Methods exist to extract this information from survival curves in reports of randomised controlled trials. However, it is unclear whether these methods give unbiased estimates when applied to non-randomised studies (NRS). Objectives: To compare estimates of log HRs extracted from published survival curves to reported adjusted log HRs in NRS. Methods: Data were extracted from NRS retrieved for a systematic review of surgical interventions for localised renal cell carcinoma. Survival probabilities were extracted from survival curve graphs using digital imaging software independently by two reviewers. Estimates of log HRs were then calculated from extracted data using methods outlined in Parmar (1998) and Williamson (2002). These estimated log HRs were compared to reported log HRs derived from models that attempted to adjust for potential selection bias. Results: There were five studies that reported both a survival curve and adjusted HR. Of these only one estimated log HR was close to the adjusted log HR. Four NRS had estimated log HR that were different to the reported adjusted log HR and the direction difference was inconsistent (differences on the log HR scale -0.16, 0.26, 0.27 and 0.65). Conclusions: In this small case study, it was empirically evident that using these methods to extract log HR from survival curves in NRS resulted in biased estimates when compared to adjusted log HR from more suitable models. In this example the potentially biased estimates of HRs was one of several reasons against a formal meta-analysis using such data. In a wider context, if reports of NRS do not include an appropriately estimated log HR (and/or information to derive its variance), reviewers should beware of the pitfalls of extracting this information from survival curves. References Parmar MK, Torri V & Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Statistics in Medicine 1998; 17:2815-34 Williamson P, Tudur Smith C, Hutton JL & Marson AG. Aggregate data meta-analysis with time-to-event outcomes. Statistics in Medicine 2002; 21:3337-51",
author = "Mari Imamura and Steven MacLennan and P Dahm and Molly Nueberger and Omar, {Muhammad Imran} and MacLennan, {Graeme Stewart}",
year = "2011",
month = "10",
language = "English",
note = "Cochrane Colloquium 2011 ; Conference date: 18-10-2011 Through 22-10-2011",

}

TY - CONF

T1 - The hazards of extracting hazard ratios from survival curves for meta-analyses in observational studies

AU - Imamura, Mari

AU - MacLennan, Steven

AU - Dahm, P

AU - Nueberger, Molly

AU - Omar, Muhammad Imran

AU - MacLennan, Graeme Stewart

PY - 2011/10

Y1 - 2011/10

N2 - Background: Meta-analyses of time-to-event data require the log hazard ratio (HR) and its variance but these data are frequently not reported. Methods exist to extract this information from survival curves in reports of randomised controlled trials. However, it is unclear whether these methods give unbiased estimates when applied to non-randomised studies (NRS). Objectives: To compare estimates of log HRs extracted from published survival curves to reported adjusted log HRs in NRS. Methods: Data were extracted from NRS retrieved for a systematic review of surgical interventions for localised renal cell carcinoma. Survival probabilities were extracted from survival curve graphs using digital imaging software independently by two reviewers. Estimates of log HRs were then calculated from extracted data using methods outlined in Parmar (1998) and Williamson (2002). These estimated log HRs were compared to reported log HRs derived from models that attempted to adjust for potential selection bias. Results: There were five studies that reported both a survival curve and adjusted HR. Of these only one estimated log HR was close to the adjusted log HR. Four NRS had estimated log HR that were different to the reported adjusted log HR and the direction difference was inconsistent (differences on the log HR scale -0.16, 0.26, 0.27 and 0.65). Conclusions: In this small case study, it was empirically evident that using these methods to extract log HR from survival curves in NRS resulted in biased estimates when compared to adjusted log HR from more suitable models. In this example the potentially biased estimates of HRs was one of several reasons against a formal meta-analysis using such data. In a wider context, if reports of NRS do not include an appropriately estimated log HR (and/or information to derive its variance), reviewers should beware of the pitfalls of extracting this information from survival curves. References Parmar MK, Torri V & Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Statistics in Medicine 1998; 17:2815-34 Williamson P, Tudur Smith C, Hutton JL & Marson AG. Aggregate data meta-analysis with time-to-event outcomes. Statistics in Medicine 2002; 21:3337-51

AB - Background: Meta-analyses of time-to-event data require the log hazard ratio (HR) and its variance but these data are frequently not reported. Methods exist to extract this information from survival curves in reports of randomised controlled trials. However, it is unclear whether these methods give unbiased estimates when applied to non-randomised studies (NRS). Objectives: To compare estimates of log HRs extracted from published survival curves to reported adjusted log HRs in NRS. Methods: Data were extracted from NRS retrieved for a systematic review of surgical interventions for localised renal cell carcinoma. Survival probabilities were extracted from survival curve graphs using digital imaging software independently by two reviewers. Estimates of log HRs were then calculated from extracted data using methods outlined in Parmar (1998) and Williamson (2002). These estimated log HRs were compared to reported log HRs derived from models that attempted to adjust for potential selection bias. Results: There were five studies that reported both a survival curve and adjusted HR. Of these only one estimated log HR was close to the adjusted log HR. Four NRS had estimated log HR that were different to the reported adjusted log HR and the direction difference was inconsistent (differences on the log HR scale -0.16, 0.26, 0.27 and 0.65). Conclusions: In this small case study, it was empirically evident that using these methods to extract log HR from survival curves in NRS resulted in biased estimates when compared to adjusted log HR from more suitable models. In this example the potentially biased estimates of HRs was one of several reasons against a formal meta-analysis using such data. In a wider context, if reports of NRS do not include an appropriately estimated log HR (and/or information to derive its variance), reviewers should beware of the pitfalls of extracting this information from survival curves. References Parmar MK, Torri V & Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Statistics in Medicine 1998; 17:2815-34 Williamson P, Tudur Smith C, Hutton JL & Marson AG. Aggregate data meta-analysis with time-to-event outcomes. Statistics in Medicine 2002; 21:3337-51

M3 - Paper

ER -