Experimental design and reporting standards for improving the internal validity of pre-clinical studies in the field of pain: Consensus of the IMI-Europain consortium

K L Knopp, C Stenfors, C Baastrup, A W Bannon, M. Calvo, O Caspani, G. Currie, N B Finnerup, W Huang, J D Kennedy, I Lefevre, I Machin, M Macleod, H Rees, A.S.C. Rice, K Rutten, M Segerdahl, J Serra, R Wodarski, O-G Berge & 1 others R-D Treede

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Background and aims

Pain is a subjective experience, and as such, pre-clinical models of human pain are highly simplified representations of clinical features. These models are nevertheless critical for the delivery of novel analgesics for human pain, providing pharmacodynamic measurements of activity and, where possible, on-target confirmation of that activity. It has, however, been suggested that at least 50% of all pre-clinical data, independent of discipline, cannot be replicated. Additionally, the paucity of “negative” data in the public domain indicates a publication bias, and significantly impacts the interpretation of failed attempts to replicate published findings. Evidence suggests that systematic biases in experimental design and conduct and insufficiencies in reporting play significant roles in poor reproducibility across pre-clinical studies. It then follows that recommendations on how to improve these factors are warranted.

Methods

Members of Europain, a pain research consortium funded by the European Innovative Medicines Initiative (IMI), developed internal recommendations on how to improve the reliability of pre-clinical studies between laboratories. This guidance is focused on two aspects: experimental design and conduct, and study reporting.

Results

Minimum requirements for experimental design and conduct were agreed upon across the dimensions of animal characteristics, sample size calculations, inclusion and exclusion criteria, random allocation to groups, allocation concealment, and blinded assessment of outcome. Building upon the Animals in Research: Reporting in vivo Experiments (ARRIVE) guidelines, reporting standards were developed for pre-clinical studies of pain. These include specific recommendations for reporting on ethical issues, experimental design and conduct, and data analysis and interpretation. Key principles such as sample size calculation, a priori definition of a primary efficacy measure, randomization, allocation concealments, and blinding are discussed. In addition, considerations of how stress and normal rodent physiology impact outcome of analgesic drug studies are considered. Flow diagrams are standard requirements in all clinical trials, and flow diagrams for preclinical trials, which describe number of animals included/excluded, and reasons for exclusion are proposed. Creation of a trial registry for pre-clinical studies focused on drug development in order to estimate possible publication bias is discussed.

Conclusions

More systematic research is needed to analyze how inadequate internal validity and/or experimental bias may impact reproducibility across pre-clinical pain studies. Addressing the potential threats to internal validity and the sources of experimental biases, as well as increasing the transparency in reporting, are likely to improve preclinical research broadly by ensuring relevant progress is made in advancing the knowledge of chronic pain pathophysiology and identifying novel analgesics.

Implications

We are now disseminating these Europain processes for discussion in the wider pain research community. Any benefit from these guidelines will be dependent on acceptance and disciplined implementation across pre-clinical laboratories, funding agencies and journal editors, but it is anticipated that these guidelines will be a first step towards improving scientific rigor across the field of pre-clinical pain research.
Original languageEnglish
Pages (from-to)58-70
Number of pages13
JournalScandinavian Journal of Pain
Volume7
Early online date10 Mar 2015
DOIs
Publication statusPublished - Apr 2015

Fingerprint

Consensus
Research Design
Pain
Analgesics
Publication Bias
Research
Guidelines
Random Allocation
Sample Size
Public Sector
Clinical Studies
Ethics
Chronic Pain
Registries
Rodentia
Outcome Assessment (Health Care)
Clinical Trials
Pharmaceutical Preparations

Keywords

  • reproducibility
  • experimental bias
  • pre-clinical pain models
  • innovative medicines
  • initiative

Cite this

Experimental design and reporting standards for improving the internal validity of pre-clinical studies in the field of pain : Consensus of the IMI-Europain consortium. / Knopp, K L ; Stenfors, C; Baastrup, C; Bannon, A W; Calvo, M.; Caspani, O; Currie, G.; Finnerup, N B; Huang, W; Kennedy, J D ; Lefevre, I; Machin, I; Macleod, M; Rees, H; Rice, A.S.C.; Rutten, K; Segerdahl, M; Serra, J; Wodarski, R; Berge, O-G; Treede, R-D.

In: Scandinavian Journal of Pain, Vol. 7, 04.2015, p. 58-70.

Research output: Contribution to journalArticle

Knopp, KL, Stenfors, C, Baastrup, C, Bannon, AW, Calvo, M, Caspani, O, Currie, G, Finnerup, NB, Huang, W, Kennedy, JD, Lefevre, I, Machin, I, Macleod, M, Rees, H, Rice, ASC, Rutten, K, Segerdahl, M, Serra, J, Wodarski, R, Berge, O-G & Treede, R-D 2015, 'Experimental design and reporting standards for improving the internal validity of pre-clinical studies in the field of pain: Consensus of the IMI-Europain consortium' Scandinavian Journal of Pain, vol. 7, pp. 58-70. https://doi.org/10.1016/j.sjpain.2015.01.006
Knopp, K L ; Stenfors, C ; Baastrup, C ; Bannon, A W ; Calvo, M. ; Caspani, O ; Currie, G. ; Finnerup, N B ; Huang, W ; Kennedy, J D ; Lefevre, I ; Machin, I ; Macleod, M ; Rees, H ; Rice, A.S.C. ; Rutten, K ; Segerdahl, M ; Serra, J ; Wodarski, R ; Berge, O-G ; Treede, R-D. / Experimental design and reporting standards for improving the internal validity of pre-clinical studies in the field of pain : Consensus of the IMI-Europain consortium. In: Scandinavian Journal of Pain. 2015 ; Vol. 7. pp. 58-70.
@article{5cb0b7e3d798458e82a0f5e1f8fcfe23,
title = "Experimental design and reporting standards for improving the internal validity of pre-clinical studies in the field of pain: Consensus of the IMI-Europain consortium",
abstract = "Background and aimsPain is a subjective experience, and as such, pre-clinical models of human pain are highly simplified representations of clinical features. These models are nevertheless critical for the delivery of novel analgesics for human pain, providing pharmacodynamic measurements of activity and, where possible, on-target confirmation of that activity. It has, however, been suggested that at least 50{\%} of all pre-clinical data, independent of discipline, cannot be replicated. Additionally, the paucity of “negative” data in the public domain indicates a publication bias, and significantly impacts the interpretation of failed attempts to replicate published findings. Evidence suggests that systematic biases in experimental design and conduct and insufficiencies in reporting play significant roles in poor reproducibility across pre-clinical studies. It then follows that recommendations on how to improve these factors are warranted.MethodsMembers of Europain, a pain research consortium funded by the European Innovative Medicines Initiative (IMI), developed internal recommendations on how to improve the reliability of pre-clinical studies between laboratories. This guidance is focused on two aspects: experimental design and conduct, and study reporting.ResultsMinimum requirements for experimental design and conduct were agreed upon across the dimensions of animal characteristics, sample size calculations, inclusion and exclusion criteria, random allocation to groups, allocation concealment, and blinded assessment of outcome. Building upon the Animals in Research: Reporting in vivo Experiments (ARRIVE) guidelines, reporting standards were developed for pre-clinical studies of pain. These include specific recommendations for reporting on ethical issues, experimental design and conduct, and data analysis and interpretation. Key principles such as sample size calculation, a priori definition of a primary efficacy measure, randomization, allocation concealments, and blinding are discussed. In addition, considerations of how stress and normal rodent physiology impact outcome of analgesic drug studies are considered. Flow diagrams are standard requirements in all clinical trials, and flow diagrams for preclinical trials, which describe number of animals included/excluded, and reasons for exclusion are proposed. Creation of a trial registry for pre-clinical studies focused on drug development in order to estimate possible publication bias is discussed.ConclusionsMore systematic research is needed to analyze how inadequate internal validity and/or experimental bias may impact reproducibility across pre-clinical pain studies. Addressing the potential threats to internal validity and the sources of experimental biases, as well as increasing the transparency in reporting, are likely to improve preclinical research broadly by ensuring relevant progress is made in advancing the knowledge of chronic pain pathophysiology and identifying novel analgesics.ImplicationsWe are now disseminating these Europain processes for discussion in the wider pain research community. Any benefit from these guidelines will be dependent on acceptance and disciplined implementation across pre-clinical laboratories, funding agencies and journal editors, but it is anticipated that these guidelines will be a first step towards improving scientific rigor across the field of pre-clinical pain research.",
keywords = "reproducibility, experimental bias, pre-clinical pain models, innovative medicines, initiative",
author = "Knopp, {K L} and C Stenfors and C Baastrup and Bannon, {A W} and M. Calvo and O Caspani and G. Currie and Finnerup, {N B} and W Huang and Kennedy, {J D} and I Lefevre and I Machin and M Macleod and H Rees and A.S.C. Rice and K Rutten and M Segerdahl and J Serra and R Wodarski and O-G Berge and R-D Treede",
year = "2015",
month = "4",
doi = "10.1016/j.sjpain.2015.01.006",
language = "English",
volume = "7",
pages = "58--70",
journal = "Scandinavian Journal of Pain",

}

TY - JOUR

T1 - Experimental design and reporting standards for improving the internal validity of pre-clinical studies in the field of pain

T2 - Consensus of the IMI-Europain consortium

AU - Knopp, K L

AU - Stenfors, C

AU - Baastrup, C

AU - Bannon, A W

AU - Calvo, M.

AU - Caspani, O

AU - Currie, G.

AU - Finnerup, N B

AU - Huang, W

AU - Kennedy, J D

AU - Lefevre, I

AU - Machin, I

AU - Macleod, M

AU - Rees, H

AU - Rice, A.S.C.

AU - Rutten, K

AU - Segerdahl, M

AU - Serra, J

AU - Wodarski, R

AU - Berge, O-G

AU - Treede, R-D

PY - 2015/4

Y1 - 2015/4

N2 - Background and aimsPain is a subjective experience, and as such, pre-clinical models of human pain are highly simplified representations of clinical features. These models are nevertheless critical for the delivery of novel analgesics for human pain, providing pharmacodynamic measurements of activity and, where possible, on-target confirmation of that activity. It has, however, been suggested that at least 50% of all pre-clinical data, independent of discipline, cannot be replicated. Additionally, the paucity of “negative” data in the public domain indicates a publication bias, and significantly impacts the interpretation of failed attempts to replicate published findings. Evidence suggests that systematic biases in experimental design and conduct and insufficiencies in reporting play significant roles in poor reproducibility across pre-clinical studies. It then follows that recommendations on how to improve these factors are warranted.MethodsMembers of Europain, a pain research consortium funded by the European Innovative Medicines Initiative (IMI), developed internal recommendations on how to improve the reliability of pre-clinical studies between laboratories. This guidance is focused on two aspects: experimental design and conduct, and study reporting.ResultsMinimum requirements for experimental design and conduct were agreed upon across the dimensions of animal characteristics, sample size calculations, inclusion and exclusion criteria, random allocation to groups, allocation concealment, and blinded assessment of outcome. Building upon the Animals in Research: Reporting in vivo Experiments (ARRIVE) guidelines, reporting standards were developed for pre-clinical studies of pain. These include specific recommendations for reporting on ethical issues, experimental design and conduct, and data analysis and interpretation. Key principles such as sample size calculation, a priori definition of a primary efficacy measure, randomization, allocation concealments, and blinding are discussed. In addition, considerations of how stress and normal rodent physiology impact outcome of analgesic drug studies are considered. Flow diagrams are standard requirements in all clinical trials, and flow diagrams for preclinical trials, which describe number of animals included/excluded, and reasons for exclusion are proposed. Creation of a trial registry for pre-clinical studies focused on drug development in order to estimate possible publication bias is discussed.ConclusionsMore systematic research is needed to analyze how inadequate internal validity and/or experimental bias may impact reproducibility across pre-clinical pain studies. Addressing the potential threats to internal validity and the sources of experimental biases, as well as increasing the transparency in reporting, are likely to improve preclinical research broadly by ensuring relevant progress is made in advancing the knowledge of chronic pain pathophysiology and identifying novel analgesics.ImplicationsWe are now disseminating these Europain processes for discussion in the wider pain research community. Any benefit from these guidelines will be dependent on acceptance and disciplined implementation across pre-clinical laboratories, funding agencies and journal editors, but it is anticipated that these guidelines will be a first step towards improving scientific rigor across the field of pre-clinical pain research.

AB - Background and aimsPain is a subjective experience, and as such, pre-clinical models of human pain are highly simplified representations of clinical features. These models are nevertheless critical for the delivery of novel analgesics for human pain, providing pharmacodynamic measurements of activity and, where possible, on-target confirmation of that activity. It has, however, been suggested that at least 50% of all pre-clinical data, independent of discipline, cannot be replicated. Additionally, the paucity of “negative” data in the public domain indicates a publication bias, and significantly impacts the interpretation of failed attempts to replicate published findings. Evidence suggests that systematic biases in experimental design and conduct and insufficiencies in reporting play significant roles in poor reproducibility across pre-clinical studies. It then follows that recommendations on how to improve these factors are warranted.MethodsMembers of Europain, a pain research consortium funded by the European Innovative Medicines Initiative (IMI), developed internal recommendations on how to improve the reliability of pre-clinical studies between laboratories. This guidance is focused on two aspects: experimental design and conduct, and study reporting.ResultsMinimum requirements for experimental design and conduct were agreed upon across the dimensions of animal characteristics, sample size calculations, inclusion and exclusion criteria, random allocation to groups, allocation concealment, and blinded assessment of outcome. Building upon the Animals in Research: Reporting in vivo Experiments (ARRIVE) guidelines, reporting standards were developed for pre-clinical studies of pain. These include specific recommendations for reporting on ethical issues, experimental design and conduct, and data analysis and interpretation. Key principles such as sample size calculation, a priori definition of a primary efficacy measure, randomization, allocation concealments, and blinding are discussed. In addition, considerations of how stress and normal rodent physiology impact outcome of analgesic drug studies are considered. Flow diagrams are standard requirements in all clinical trials, and flow diagrams for preclinical trials, which describe number of animals included/excluded, and reasons for exclusion are proposed. Creation of a trial registry for pre-clinical studies focused on drug development in order to estimate possible publication bias is discussed.ConclusionsMore systematic research is needed to analyze how inadequate internal validity and/or experimental bias may impact reproducibility across pre-clinical pain studies. Addressing the potential threats to internal validity and the sources of experimental biases, as well as increasing the transparency in reporting, are likely to improve preclinical research broadly by ensuring relevant progress is made in advancing the knowledge of chronic pain pathophysiology and identifying novel analgesics.ImplicationsWe are now disseminating these Europain processes for discussion in the wider pain research community. Any benefit from these guidelines will be dependent on acceptance and disciplined implementation across pre-clinical laboratories, funding agencies and journal editors, but it is anticipated that these guidelines will be a first step towards improving scientific rigor across the field of pre-clinical pain research.

KW - reproducibility

KW - experimental bias

KW - pre-clinical pain models

KW - innovative medicines

KW - initiative

U2 - 10.1016/j.sjpain.2015.01.006

DO - 10.1016/j.sjpain.2015.01.006

M3 - Article

VL - 7

SP - 58

EP - 70

JO - Scandinavian Journal of Pain

JF - Scandinavian Journal of Pain

ER -