Single case research in neuropsychology

a comparison of five forms of t-test for comparing a case to controls

John R. Crawford, Paul H. Garthwaite

Research output: Contribution to journalArticle

53 Citations (Scopus)

Abstract

Five inferential methods employed in single-case studies to compare a case to controls are examined; all of these make use of a t-distribution. It is shown that three of these ostensibly different methods are in fact strictly equivalent and are not fit for purpose; they are associated with grossly inflated Type I errors (these exceed even the error rate obtained when a case’s score is converted to a z score and the latter used as a test statistic). When used as significance tests, the two remaining methods (Crawford and Howell’s method and a prediction interval method first used by Barton and colleagues) are also equivalent and achieve control of the Type I error rate (the two methods do differ however in other important aspects). A number of broader issues also arise from the present findings, namely: (a) they underline the value of accompanying significance test results with the effect size for the difference between a case and controls, (b) they suggest that less care is often taken over statistical methods than over other aspects of single-case studies, and (c) they indicate that some neuropsychologists have a distorted conception of the nature of hypothesis testing in single-case research (it is argued that this may stem from a failure to distinguish between group studies and single-case studies).
Original languageEnglish
Pages (from-to)1009-1016
Number of pages8
JournalCortex
Volume48
Issue number8
Early online date23 Jul 2011
DOIs
Publication statusPublished - Sep 2012

Fingerprint

Neuropsychology
Research

Keywords

  • single-case methods
  • case-controls design
  • t-tests
  • neuropsychological methods

Cite this

Single case research in neuropsychology : a comparison of five forms of t-test for comparing a case to controls. / Crawford, John R.; Garthwaite, Paul H.

In: Cortex, Vol. 48, No. 8, 09.2012, p. 1009-1016.

Research output: Contribution to journalArticle

@article{080c5a0ff9fe4b5e906ed2c679247bd7,
title = "Single case research in neuropsychology: a comparison of five forms of t-test for comparing a case to controls",
abstract = "Five inferential methods employed in single-case studies to compare a case to controls are examined; all of these make use of a t-distribution. It is shown that three of these ostensibly different methods are in fact strictly equivalent and are not fit for purpose; they are associated with grossly inflated Type I errors (these exceed even the error rate obtained when a case’s score is converted to a z score and the latter used as a test statistic). When used as significance tests, the two remaining methods (Crawford and Howell’s method and a prediction interval method first used by Barton and colleagues) are also equivalent and achieve control of the Type I error rate (the two methods do differ however in other important aspects). A number of broader issues also arise from the present findings, namely: (a) they underline the value of accompanying significance test results with the effect size for the difference between a case and controls, (b) they suggest that less care is often taken over statistical methods than over other aspects of single-case studies, and (c) they indicate that some neuropsychologists have a distorted conception of the nature of hypothesis testing in single-case research (it is argued that this may stem from a failure to distinguish between group studies and single-case studies).",
keywords = "single-case methods, case-controls design, t-tests, neuropsychological methods",
author = "Crawford, {John R.} and Garthwaite, {Paul H.}",
year = "2012",
month = "9",
doi = "10.1016/j.cortex.2011.06.021",
language = "English",
volume = "48",
pages = "1009--1016",
journal = "Cortex",
issn = "0010-9452",
publisher = "Masson SpA",
number = "8",

}

TY - JOUR

T1 - Single case research in neuropsychology

T2 - a comparison of five forms of t-test for comparing a case to controls

AU - Crawford, John R.

AU - Garthwaite, Paul H.

PY - 2012/9

Y1 - 2012/9

N2 - Five inferential methods employed in single-case studies to compare a case to controls are examined; all of these make use of a t-distribution. It is shown that three of these ostensibly different methods are in fact strictly equivalent and are not fit for purpose; they are associated with grossly inflated Type I errors (these exceed even the error rate obtained when a case’s score is converted to a z score and the latter used as a test statistic). When used as significance tests, the two remaining methods (Crawford and Howell’s method and a prediction interval method first used by Barton and colleagues) are also equivalent and achieve control of the Type I error rate (the two methods do differ however in other important aspects). A number of broader issues also arise from the present findings, namely: (a) they underline the value of accompanying significance test results with the effect size for the difference between a case and controls, (b) they suggest that less care is often taken over statistical methods than over other aspects of single-case studies, and (c) they indicate that some neuropsychologists have a distorted conception of the nature of hypothesis testing in single-case research (it is argued that this may stem from a failure to distinguish between group studies and single-case studies).

AB - Five inferential methods employed in single-case studies to compare a case to controls are examined; all of these make use of a t-distribution. It is shown that three of these ostensibly different methods are in fact strictly equivalent and are not fit for purpose; they are associated with grossly inflated Type I errors (these exceed even the error rate obtained when a case’s score is converted to a z score and the latter used as a test statistic). When used as significance tests, the two remaining methods (Crawford and Howell’s method and a prediction interval method first used by Barton and colleagues) are also equivalent and achieve control of the Type I error rate (the two methods do differ however in other important aspects). A number of broader issues also arise from the present findings, namely: (a) they underline the value of accompanying significance test results with the effect size for the difference between a case and controls, (b) they suggest that less care is often taken over statistical methods than over other aspects of single-case studies, and (c) they indicate that some neuropsychologists have a distorted conception of the nature of hypothesis testing in single-case research (it is argued that this may stem from a failure to distinguish between group studies and single-case studies).

KW - single-case methods

KW - case-controls design

KW - t-tests

KW - neuropsychological methods

U2 - 10.1016/j.cortex.2011.06.021

DO - 10.1016/j.cortex.2011.06.021

M3 - Article

VL - 48

SP - 1009

EP - 1016

JO - Cortex

JF - Cortex

SN - 0010-9452

IS - 8

ER -