Rounding, but not randomization method, non-normality, or correlation, affected baseline P-value distributions in randomized trials

TY - JOUR

T1 - Rounding, but not randomization method, non-normality, or correlation, affected baseline P-value distributions in randomized trials

AU - Bolland, Mark J

AU - Gamble, Greg D

AU - Avenell, Alison

AU - Grey, Andrew

N1 - Acknowledgements: The authors acknowledge the contribution of Professor Thomas Lumley who provided expert guidance in using the area under the curve of the cumulative distribution function of the baseline p-values for these analyses No specific funding was received for this study. MB receives salary support from the Health Research Council of New Zealand. The Health Services Research Unit is funded by the Chief Scientist Office of the Scottish Government Health and Social Care Directorates.The funders had no role in the study design; collection, analysis, and interpretation of the data; writing of the report; and in the decision to submit the paper for publication

PY - 2019/6

Y1 - 2019/6

N2 - OBJECTIVE: To investigate whether comparing observed with expected p-value distributions for baseline continuous variables in randomised controlled trials (RCTs) might be limited by randomisation methods, normality and correlation of variables, or calculation of p-values from rounded summary statistics.STUDY DESIGN AND SETTING: We assessed how each factor affects differences from expected for p-value distributions and area under the curve of the cumulative distribution function (AUC-CDF) of baseline p-values in 13 RCTs and in simulations.RESULTS: The p-value distributions and AUC-CDF for variables with possible non-normal distribution and in simulations using eight different randomisation methods were consistent with the theoretical uniform distribution and AUC-CDF respectively, although stratification and minimisation produced smaller-than-expected proportions of p-values <0.10. 77% of 3813 pairwise correlations between baseline variables in the 13 individual RCTs were between -0.2 and 0.2. P-value distribution and AUC-CDF remained consistent with the uniform distribution in simulations with incrementally increasing correlation strength. The p-value distributions calculated from rounded summary statistics were not uniform, but expected distributions could be empirically generated.CONCLUSIONS: Randomisation methods, non-normality and strength of correlation of baseline variables did not have important effects on baseline p-value distribution or AUC-CDF, but baseline p-values calculated from rounded summary statistics are non-uniformly distributed.

AB - OBJECTIVE: To investigate whether comparing observed with expected p-value distributions for baseline continuous variables in randomised controlled trials (RCTs) might be limited by randomisation methods, normality and correlation of variables, or calculation of p-values from rounded summary statistics.STUDY DESIGN AND SETTING: We assessed how each factor affects differences from expected for p-value distributions and area under the curve of the cumulative distribution function (AUC-CDF) of baseline p-values in 13 RCTs and in simulations.RESULTS: The p-value distributions and AUC-CDF for variables with possible non-normal distribution and in simulations using eight different randomisation methods were consistent with the theoretical uniform distribution and AUC-CDF respectively, although stratification and minimisation produced smaller-than-expected proportions of p-values <0.10. 77% of 3813 pairwise correlations between baseline variables in the 13 individual RCTs were between -0.2 and 0.2. P-value distribution and AUC-CDF remained consistent with the uniform distribution in simulations with incrementally increasing correlation strength. The p-value distributions calculated from rounded summary statistics were not uniform, but expected distributions could be empirically generated.CONCLUSIONS: Randomisation methods, non-normality and strength of correlation of baseline variables did not have important effects on baseline p-value distribution or AUC-CDF, but baseline p-values calculated from rounded summary statistics are non-uniformly distributed.

KW - Statistical Methods

KW - Research Integrity

KW - P-values

KW - Correlation

KW - Randomisation

KW - Rounding

KW - Research integrity

KW - Statistical methods

UR - http://www.scopus.com/inward/record.url?scp=85063631801&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/rounding-not-randomization-method-nonnormality-correlation-affected-baseline-pvalue-distributions-ra

U2 - 10.1016/j.jclinepi.2019.03.001

DO - 10.1016/j.jclinepi.2019.03.001

M3 - Article

VL - 110

SP - 50

EP - 62

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -