Using the Revised Cardiac Risk Index to predict major postoperative events for people with kidney failure: An external validation and update

Tyrone Gorden  Harrison; Brenda R  Hemmelgarn; Matthew T James; Simon Sawhney; Ngan Lam; Shannon M Ruzycki; Todd Allen  Wilson; Paul E.  Ronksley

doi:10.1016/j.cjco.2022.07.008

Using the Revised Cardiac Risk Index to predict major postoperative events for people with kidney failure: An external validation and update

Tyrone Gorden Harrison, Brenda R Hemmelgarn , Matthew T James, Simon Sawhney, Ngan Lam, Shannon M Ruzycki, Todd Allen Wilson , Paul E. Ronksley^* (Corresponding Author)

^*Corresponding author for this work

University of Calgary

Research output: Contribution to journal › Article › peer-review

2 Citations (Scopus)

Abstract

Background
People with kidney failure have high risk of postoperative morbidity and mortality. Although the Revised Cardiac Risk Index (RCRI) is used to estimate the risk of major postoperative events, it has not been validated in this population. We aimed to externally validate the RCRI and determine whether updating the model improved predictions for people with kidney failure.

Methods
We derived a retrospective, population-based cohort of adults with kidney failure (maintenance dialysis or sustained estimated glomerular filtration rate [eGFR] <15 mL/min/1.73 m2) who had surgery in Alberta, Canada between 2005 and 2019. We categorized participants based on RCRI variables and assigned risk estimates of death or major cardiac events, and then estimated predictive performance. We re-estimated the coefficients for each RCRI variable and internally validated the updated model. Net benefit was estimated with decision curve analysis.

Results
After 38,541 surgeries, 1,204 (3.1%) events occurred. The estimated C-statistic for the original RCRI was 0.64 (95% confidence interval [CI]: 0.62, 0.65). Examination of calibration revealed significant risk overestimation. In the re-estimated RCRI model, discrimination was marginally different (C-statistic 0.67 [95%CI 0.66, 0.69]), though calibration was improved. There was net benefit when examined with decision curve analysis, while the original RCRI was associated with harm.

Conclusions
The RCRI performed poorly in a Canadian kidney failure cohort and significantly overestimated risk, suggesting RCRI use in similar kidney failure populations should be limited. A re-estimated kidney failure specific RCRI may be promising, though needs external validation. Novel perioperative models for this population are urgently needed.

Original language	English
Pages (from-to)	905-912
Number of pages	8
Journal	CJC Open
Volume	4
Issue number	10
Early online date	14 Jul 2022
DOIs	https://doi.org/10.1016/j.cjco.2022.07.008
Publication status	Published - Oct 2022

Bibliographical note

Funding Information:
T.G.H. is supported by a Kidney Research Scientist Core Education and National Training Program postdoctoral fellowship (cosponsored by the Kidney Foundation of Canada and Canadian Institutes of Health Research) and the Clinician Investigator Program at the University of Calgary. These funding sources had no role in study design, data collection, analysis, reporting, or the decision to submit for publication.

Funding Information:
Ethics Statement: We followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) checklist19 for prediction-model validation (Supplemental Table S1) and were granted ethics approval by the University of Calgary and the University of Alberta.Preoperative risk-prediction tools that are used to predict risk of perioperative death and CV events, and are supported by North American guidelines, include the revised cardiac risk index (RCRI),5 the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) tool,6,7 and the National Surgical Quality Improvement Program Myocardial Infarction or Cardiac Arrest (NSQIP MICA) tool.8 The RCRI has been recommended over others for use in Canada for all adults over the age of 45 years, and for those aged 18-45 years with CV disease, who are undergoing elective, noncardiac surgery.3 The RCRI incorporates 6 criteria based on surgical and comorbidity characteristics of the patient and derives an estimated probability of postoperative myocardial infarction, cardiac arrest, or death.5 Additionally, the RCRI is used to guide perioperative decision-making.3The Alberta Kidney Disease Network database includes person-level linkages of administrative health data, laboratory data, prescription information, and kidney disease-specific data from the province of Alberta, Canada.17 Alberta has approximately 4.4 million residents, and with universal public health insurance, health data capture is near complete.17,18 From this database, we derived a retrospective cohort of adults with kidney failure who underwent ambulatory or inpatient surgery. We used this cohort to externally validate and examine the performance of the RCRI for this population. We followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) checklist19 for prediction-model validation (Supplemental Table S1) and were granted ethics approval by the University of Calgary and the University of Alberta.

Keywords

kidney failure
perioperative
surgery
Risk prediction
RCRI
validation

Access to Document

10.1016/j.cjco.2022.07.008Licence: CC BY-NC-ND

Harrison_etal_CJCO_UsingtheRevisedCardiac_VOR
This article is licensed under a Creative Commons Attribution Non-Commercial No Derivatives 4.0 International License, which permits non-commercial use, sharing and reproduction in any medium or format, as long as you do not change or modify the original and give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. https://creativecommons.org/licenses/by-nc-nd/4.0/
Final published version, 630 KBLicence: CC BY-NC-ND

Cite this

@article{21945448bae04104b67f351a66570e0b,

title = "Using the Revised Cardiac Risk Index to predict major postoperative events for people with kidney failure: An external validation and update",

abstract = "BackgroundPeople with kidney failure have high risk of postoperative morbidity and mortality. Although the Revised Cardiac Risk Index (RCRI) is used to estimate the risk of major postoperative events, it has not been validated in this population. We aimed to externally validate the RCRI and determine whether updating the model improved predictions for people with kidney failure.MethodsWe derived a retrospective, population-based cohort of adults with kidney failure (maintenance dialysis or sustained estimated glomerular filtration rate [eGFR] <15 mL/min/1.73 m2) who had surgery in Alberta, Canada between 2005 and 2019. We categorized participants based on RCRI variables and assigned risk estimates of death or major cardiac events, and then estimated predictive performance. We re-estimated the coefficients for each RCRI variable and internally validated the updated model. Net benefit was estimated with decision curve analysis.ResultsAfter 38,541 surgeries, 1,204 (3.1%) events occurred. The estimated C-statistic for the original RCRI was 0.64 (95% confidence interval [CI]: 0.62, 0.65). Examination of calibration revealed significant risk overestimation. In the re-estimated RCRI model, discrimination was marginally different (C-statistic 0.67 [95%CI 0.66, 0.69]), though calibration was improved. There was net benefit when examined with decision curve analysis, while the original RCRI was associated with harm.ConclusionsThe RCRI performed poorly in a Canadian kidney failure cohort and significantly overestimated risk, suggesting RCRI use in similar kidney failure populations should be limited. A re-estimated kidney failure specific RCRI may be promising, though needs external validation. Novel perioperative models for this population are urgently needed.",

keywords = "kidney failure, perioperative, surgery, Risk prediction, RCRI, validation",

author = "Harrison, {Tyrone Gorden} and Hemmelgarn, {Brenda R} and James, {Matthew T} and Simon Sawhney and Ngan Lam and Ruzycki, {Shannon M} and Wilson, {Todd Allen} and Ronksley, {Paul E.}",

note = "Funding Information: T.G.H. is supported by a Kidney Research Scientist Core Education and National Training Program postdoctoral fellowship (cosponsored by the Kidney Foundation of Canada and Canadian Institutes of Health Research) and the Clinician Investigator Program at the University of Calgary. These funding sources had no role in study design, data collection, analysis, reporting, or the decision to submit for publication. Funding Information: Ethics Statement: We followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) checklist19 for prediction-model validation (Supplemental Table S1) and were granted ethics approval by the University of Calgary and the University of Alberta.Preoperative risk-prediction tools that are used to predict risk of perioperative death and CV events, and are supported by North American guidelines, include the revised cardiac risk index (RCRI),5 the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) tool,6,7 and the National Surgical Quality Improvement Program Myocardial Infarction or Cardiac Arrest (NSQIP MICA) tool.8 The RCRI has been recommended over others for use in Canada for all adults over the age of 45 years, and for those aged 18-45 years with CV disease, who are undergoing elective, noncardiac surgery.3 The RCRI incorporates 6 criteria based on surgical and comorbidity characteristics of the patient and derives an estimated probability of postoperative myocardial infarction, cardiac arrest, or death.5 Additionally, the RCRI is used to guide perioperative decision-making.3The Alberta Kidney Disease Network database includes person-level linkages of administrative health data, laboratory data, prescription information, and kidney disease-specific data from the province of Alberta, Canada.17 Alberta has approximately 4.4 million residents, and with universal public health insurance, health data capture is near complete.17,18 From this database, we derived a retrospective cohort of adults with kidney failure who underwent ambulatory or inpatient surgery. We used this cohort to externally validate and examine the performance of the RCRI for this population. We followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) checklist19 for prediction-model validation (Supplemental Table S1) and were granted ethics approval by the University of Calgary and the University of Alberta. ",

year = "2022",

month = oct,

doi = "10.1016/j.cjco.2022.07.008",

language = "English",

volume = "4",

pages = "905--912",

journal = "CJC Open",

issn = "2589-790X",

publisher = "Elsevier",

number = "10",

}

TY - JOUR

T1 - Using the Revised Cardiac Risk Index to predict major postoperative events for people with kidney failure

T2 - An external validation and update

AU - Harrison, Tyrone Gorden

AU - Hemmelgarn , Brenda R

AU - James, Matthew T

AU - Sawhney, Simon

AU - Lam, Ngan

AU - Ruzycki, Shannon M

AU - Wilson , Todd Allen

AU - Ronksley, Paul E.

N1 - Funding Information: T.G.H. is supported by a Kidney Research Scientist Core Education and National Training Program postdoctoral fellowship (cosponsored by the Kidney Foundation of Canada and Canadian Institutes of Health Research) and the Clinician Investigator Program at the University of Calgary. These funding sources had no role in study design, data collection, analysis, reporting, or the decision to submit for publication. Funding Information: Ethics Statement: We followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) checklist19 for prediction-model validation (Supplemental Table S1) and were granted ethics approval by the University of Calgary and the University of Alberta.Preoperative risk-prediction tools that are used to predict risk of perioperative death and CV events, and are supported by North American guidelines, include the revised cardiac risk index (RCRI),5 the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) tool,6,7 and the National Surgical Quality Improvement Program Myocardial Infarction or Cardiac Arrest (NSQIP MICA) tool.8 The RCRI has been recommended over others for use in Canada for all adults over the age of 45 years, and for those aged 18-45 years with CV disease, who are undergoing elective, noncardiac surgery.3 The RCRI incorporates 6 criteria based on surgical and comorbidity characteristics of the patient and derives an estimated probability of postoperative myocardial infarction, cardiac arrest, or death.5 Additionally, the RCRI is used to guide perioperative decision-making.3The Alberta Kidney Disease Network database includes person-level linkages of administrative health data, laboratory data, prescription information, and kidney disease-specific data from the province of Alberta, Canada.17 Alberta has approximately 4.4 million residents, and with universal public health insurance, health data capture is near complete.17,18 From this database, we derived a retrospective cohort of adults with kidney failure who underwent ambulatory or inpatient surgery. We used this cohort to externally validate and examine the performance of the RCRI for this population. We followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) checklist19 for prediction-model validation (Supplemental Table S1) and were granted ethics approval by the University of Calgary and the University of Alberta.

PY - 2022/10

Y1 - 2022/10

N2 - BackgroundPeople with kidney failure have high risk of postoperative morbidity and mortality. Although the Revised Cardiac Risk Index (RCRI) is used to estimate the risk of major postoperative events, it has not been validated in this population. We aimed to externally validate the RCRI and determine whether updating the model improved predictions for people with kidney failure.MethodsWe derived a retrospective, population-based cohort of adults with kidney failure (maintenance dialysis or sustained estimated glomerular filtration rate [eGFR] <15 mL/min/1.73 m2) who had surgery in Alberta, Canada between 2005 and 2019. We categorized participants based on RCRI variables and assigned risk estimates of death or major cardiac events, and then estimated predictive performance. We re-estimated the coefficients for each RCRI variable and internally validated the updated model. Net benefit was estimated with decision curve analysis.ResultsAfter 38,541 surgeries, 1,204 (3.1%) events occurred. The estimated C-statistic for the original RCRI was 0.64 (95% confidence interval [CI]: 0.62, 0.65). Examination of calibration revealed significant risk overestimation. In the re-estimated RCRI model, discrimination was marginally different (C-statistic 0.67 [95%CI 0.66, 0.69]), though calibration was improved. There was net benefit when examined with decision curve analysis, while the original RCRI was associated with harm.ConclusionsThe RCRI performed poorly in a Canadian kidney failure cohort and significantly overestimated risk, suggesting RCRI use in similar kidney failure populations should be limited. A re-estimated kidney failure specific RCRI may be promising, though needs external validation. Novel perioperative models for this population are urgently needed.

AB - BackgroundPeople with kidney failure have high risk of postoperative morbidity and mortality. Although the Revised Cardiac Risk Index (RCRI) is used to estimate the risk of major postoperative events, it has not been validated in this population. We aimed to externally validate the RCRI and determine whether updating the model improved predictions for people with kidney failure.MethodsWe derived a retrospective, population-based cohort of adults with kidney failure (maintenance dialysis or sustained estimated glomerular filtration rate [eGFR] <15 mL/min/1.73 m2) who had surgery in Alberta, Canada between 2005 and 2019. We categorized participants based on RCRI variables and assigned risk estimates of death or major cardiac events, and then estimated predictive performance. We re-estimated the coefficients for each RCRI variable and internally validated the updated model. Net benefit was estimated with decision curve analysis.ResultsAfter 38,541 surgeries, 1,204 (3.1%) events occurred. The estimated C-statistic for the original RCRI was 0.64 (95% confidence interval [CI]: 0.62, 0.65). Examination of calibration revealed significant risk overestimation. In the re-estimated RCRI model, discrimination was marginally different (C-statistic 0.67 [95%CI 0.66, 0.69]), though calibration was improved. There was net benefit when examined with decision curve analysis, while the original RCRI was associated with harm.ConclusionsThe RCRI performed poorly in a Canadian kidney failure cohort and significantly overestimated risk, suggesting RCRI use in similar kidney failure populations should be limited. A re-estimated kidney failure specific RCRI may be promising, though needs external validation. Novel perioperative models for this population are urgently needed.

KW - kidney failure

KW - perioperative

KW - surgery

KW - Risk prediction

KW - RCRI

KW - validation

U2 - 10.1016/j.cjco.2022.07.008

DO - 10.1016/j.cjco.2022.07.008

M3 - Article

C2 - 36254324

SN - 2589-790X

VL - 4

SP - 905

EP - 912

JO - CJC Open

JF - CJC Open

IS - 10

ER -

Using the Revised Cardiac Risk Index to predict major postoperative events for people with kidney failure: An external validation and update

Abstract

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this