How consistent are ratings and raters on the PCEPS? Interrater Reliability and Item Structure

Elizabeth Schmitt Freire, Robert Elliott, Graham Westwell

Research output: Contribution to conferenceUnpublished paperpeer-review

Abstract

Aim/Purpose: This study aimed to assess the reliability of the Person Centered and Experiential Psychotherapy Scale (PCEPS), a new adherence/competence measure of person-centred and experiential psychotherapies.

Design/Methodology: One-hundred twenty audio-recorded segments of therapy sessions selected from the archive of taped therapy sessions of the Strathclyde Therapy Research Centre were rated independently by two teams of three raters using the PCEPS. Six segments were systematically selected from 20 clients seen by 10 therapists (2 clients per therapist); in the first, middle, and last third of therapy; and in the first and second half of therapy sessions. Half of the segments were 10 min long and the other half were 15 min long. Six therapists were experienced therapists and four were counsellors in training. Seven of the therapists identified their work as 'person-centred’, and three of the experienced practitioners identifed their work as 'process-experiential' (although one of these was not fully trained in this approach). Three raters were qualified and experienced person-centred therapists and 3 raters were person-centred counselling trainees in their first year of training. In this paper we present the inter-item and inter-rater reliability results from this study, followed by exploratory factor analyses to study scale structure.

Results/Findings: Mean interrater alpha reliabilities for individual items varied from .68 (for Dominant or Overpowering Presence) to .86 (for Core Meaning). Average interrater reliability across the 15 items was .78, while the interrater reliability of the 15 items when averaged together was .87. Interitem reliabilities were quite high: .98 for the whole. Exploratory factor analyses revealed a 12-item facilitative relationship scale that cut across Person-centred and Experiential subscales (alpha: .98), and a nonfacilitative directiveness factor (3 items, alpha: .89).

Research Limitations: Therapy process measures in general and therapist adherence/competence measures in particular can be criticized for ignoring context and participant internal experiences, while attempting to generalize from relative brief segments of therapy to therapist performance in general. The present study would also be strengthened by comparison to non person-centred-experiential therapies.

Conclusions/Implications: The PCEPS has potential for use in RCT research as well as in counselling training and supervision, but will require further testing and validation.
Original languageEnglish
Publication statusUnpublished - 2012
Event10th Conference for the World Association for Person-Centered and Experiential Psychotherapy and Counseling - Antwerp, Belgium
Duration: 8 Jul 201212 Jul 2012

Conference

Conference10th Conference for the World Association for Person-Centered and Experiential Psychotherapy and Counseling
Country/TerritoryBelgium
CityAntwerp
Period8/07/1212/07/12

Keywords

  • person-centred therapy
  • adherence/competence
  • experiential therapy
  • measure development

Fingerprint

Dive into the research topics of 'How consistent are ratings and raters on the PCEPS? Interrater Reliability and Item Structure'. Together they form a unique fingerprint.

Cite this