Statistical validation of event predictors: A comparative study based on the field of seizure prediction

Hinnerk Feldwisch-Drentrup, Andreas Schulze-Bonhage, Jens Timmer, Bjoern Schelter

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

The prediction of events is of substantial interest in many research areas. To evaluate the performance of prediction methods, the statistical validation of these methods is of utmost importance. Here, we compare an analytical validation method to numerical approaches that are based on Monte Carlo simulations. The comparison is performed in the field of the prediction of epileptic seizures. In contrast to the analytical validation method, we found that for numerical validation methods insufficient but realistic sample sizes can lead to invalid high rates of false positive conclusions. Hence we outline necessary preconditions for sound statistical tests on above chance predictions
Original languageEnglish
Article number066704
Number of pages6
JournalPhysical Review. E, Statistical, Nonlinear and Soft Matter Physics
Volume83
Issue number6
Early online date20 Jun 2011
DOIs
Publication statusPublished - Jun 2011

Fingerprint

seizures
Comparative Study
Predictors
Prediction
predictions
statistical tests
Precondition
Statistical test
False Positive
Sample Size
Monte Carlo Simulation
acoustics
Necessary
Evaluate
simulation

Cite this

Statistical validation of event predictors : A comparative study based on the field of seizure prediction. / Feldwisch-Drentrup, Hinnerk; Schulze-Bonhage, Andreas; Timmer, Jens; Schelter, Bjoern.

In: Physical Review. E, Statistical, Nonlinear and Soft Matter Physics, Vol. 83, No. 6, 066704, 06.2011.

Research output: Contribution to journalArticle

@article{f600496f98e040199f9e8c67849347c6,
title = "Statistical validation of event predictors: A comparative study based on the field of seizure prediction",
abstract = "The prediction of events is of substantial interest in many research areas. To evaluate the performance of prediction methods, the statistical validation of these methods is of utmost importance. Here, we compare an analytical validation method to numerical approaches that are based on Monte Carlo simulations. The comparison is performed in the field of the prediction of epileptic seizures. In contrast to the analytical validation method, we found that for numerical validation methods insufficient but realistic sample sizes can lead to invalid high rates of false positive conclusions. Hence we outline necessary preconditions for sound statistical tests on above chance predictions",
author = "Hinnerk Feldwisch-Drentrup and Andreas Schulze-Bonhage and Jens Timmer and Bjoern Schelter",
year = "2011",
month = "6",
doi = "10.1103/PhysRevE.83.066704",
language = "English",
volume = "83",
journal = "Physical Review. E, Statistical, Nonlinear and Soft Matter Physics",
issn = "1539-3755",
publisher = "AMER PHYSICAL SOC",
number = "6",

}

TY - JOUR

T1 - Statistical validation of event predictors

T2 - A comparative study based on the field of seizure prediction

AU - Feldwisch-Drentrup, Hinnerk

AU - Schulze-Bonhage, Andreas

AU - Timmer, Jens

AU - Schelter, Bjoern

PY - 2011/6

Y1 - 2011/6

N2 - The prediction of events is of substantial interest in many research areas. To evaluate the performance of prediction methods, the statistical validation of these methods is of utmost importance. Here, we compare an analytical validation method to numerical approaches that are based on Monte Carlo simulations. The comparison is performed in the field of the prediction of epileptic seizures. In contrast to the analytical validation method, we found that for numerical validation methods insufficient but realistic sample sizes can lead to invalid high rates of false positive conclusions. Hence we outline necessary preconditions for sound statistical tests on above chance predictions

AB - The prediction of events is of substantial interest in many research areas. To evaluate the performance of prediction methods, the statistical validation of these methods is of utmost importance. Here, we compare an analytical validation method to numerical approaches that are based on Monte Carlo simulations. The comparison is performed in the field of the prediction of epileptic seizures. In contrast to the analytical validation method, we found that for numerical validation methods insufficient but realistic sample sizes can lead to invalid high rates of false positive conclusions. Hence we outline necessary preconditions for sound statistical tests on above chance predictions

U2 - 10.1103/PhysRevE.83.066704

DO - 10.1103/PhysRevE.83.066704

M3 - Article

VL - 83

JO - Physical Review. E, Statistical, Nonlinear and Soft Matter Physics

JF - Physical Review. E, Statistical, Nonlinear and Soft Matter Physics

SN - 1539-3755

IS - 6

M1 - 066704

ER -