Original research article
Open Access

Validation of administrative database codes for acute kidney injury in kidney transplant recipients

  • Amber O. Molnar1, 8Email author,
  • Carl van Walraven2, 3, 4,
  • Eric McArthur3,
  • Dean Fergusson2, 4,
  • Amit X. Garg3, 5, 6 and
  • Greg Knoll2, 7
Canadian Journal of Kidney Health and Disease20163:18

https://doi.org/10.1186/s40697-016-0108-7

©  Molnar et al. 2016

Received: 18 December 2015

Accepted: 8 March 2016

Published: 7 April 2016

Abstract

Background

Validation studies of acute kidney injury (AKI) diagnostic codes performed in the general population have shown poor sensitivity, but the accuracy of such codes in the kidney transplant population remains unknown.

Objective

The objective of this study is to determine the accuracy of AKI diagnostic codes in kidney transplant recipients. We hypothesized that the sensitivity of diagnostic codes would be significantly greater in the kidney transplant population since these patients are closely followed by nephrologists and are more likely to have serum creatinine measured.

Design

The design is a population-based retrospective cohort study using healthcare administrative and laboratory databases.

Setting

The setting is in Southwestern Ontario and Ottawa, Ontario, Canada, from 2003 to 2012.

Patients

We included first-time kidney transplant recipients admitted to hospital for whom serum creatinine was measured in hospital and within 6 months prior (n = 524).

Methods

Patients meeting the Acute Kidney Injury Network (AKIN) classification serum creatinine change criteria were classified as having AKI. We determined the sensitivity, specificity, and negative and positive predictive values for the ICD-10-CA code for AKI when present as an admission diagnosis, most responsible diagnosis, or any diagnosis compared to a reference standard of AKI defined by the AKIN criteria (stage 1 or greater, stage 2 or greater, or stage 3).

Results

Forty-five percent of included kidney transplant patients had a diagnosis of AKI. The most sensitive coding algorithm (reference standard AKIN stage 2 or greater, ICD-10 code present as any diagnosis) had a sensitivity of 42.1 % (95 % CI 31.7, 53.3), a specificity of 90.6 % (95 % CI 87.6, 93.0), and a positive likelihood ratio of 4.5. The median (IQR) rise in serum creatinine from baseline in patients with and without AKI codes was 104 (57 to 158) μmol/L and 16 (−3 to 41) μmol/L, respectively (Mann-Whitney test, p < 0.0001).

Limitations

The low sensitivity of the AKI code may be due to an alternative diagnosis of acute rejection being assigned in certain cases. The cause of AKI could not be determined.

Conclusions

Similar to the general population, the ICD-10 N17x code misses many kidney transplant patients with AKI during their hospitalization. This makes the code unusable for studying the incidence and consequences of AKI in hospitalized kidney transplant patients.

Keywords

Acute kidney injury Transplant Codes Validity Databases

Abrégé

Mise en contexte

Les études de validation portant sur les codes diagnostiques pour l’identification de l’insuffisance rénale aiguë (IRA) menées dans la population générale ont révélé une piètre sensibilité de détection de ceux-ci. Toutefois, leur fiabilité chez les patients ayant subi une greffe du rein reste à démontrer.

Objectif de l’étude

Cette étude visait à évaluer la précision des codes de diagnostic de l’IRA chez les patients ayant reçu une greffe rénale. Le postulat de départ était que la sensibilité de ces codes serait nettement supérieure au sein de cette population puisque ces patients bénéficient d’un suivi par un néphrologue et sont plus susceptibles de voir leur taux de créatinine mesuré régulièrement.

Cadre et type d’étude

Il s’agit d’une étude de cohorte rétrospective, représentative de la population. Elle a été réalisée entre 2003 et 2012, à Ottawa et dans le sud-ouest de l’Ontario, à partir de bases de données de laboratoire.

Participants

Un total de 524 receveurs d’une première greffe rénale ayant été hospitalisés et dont la créatinine a été mesurée au moment de l’hospitalisation et dans les 6 mois précédant celle-ci.

Méthodologie

Les patients dont le taux de créatinine correspondait aux stades de classification de l’IRA du Acute Kidney Injury Network (AKIN) ont été considérés en insuffisance rénale aiguë. Des valeurs de spécificité et de sensibilité ainsi que des valeurs de prédiction positives et négatives pour le code ICD-10-CA en situation d’IRA ont été déterminées lorsque l’IRA constituait le diagnostic d’admission, le diagnostic le plus probable mis en cause, ou n’importe lequel des diagnostics évoqués, en comparaison avec les standards de référence définis par les critères de l’AKIN (stade 1 ou supérieur, stade 2 ou supérieur, ou stade 3).

Résultats

Au total, 45 % des patients admis dans cette étude ont reçu un diagnostic d’IRA. L’algorithme de codage le plus sensible [stade 2 en référence aux standards AKIN avec pour tout diagnostic la présence du code ICD-10-CA] présentait une sensibilité de 42,1 % [95 % IC 31,7 à 53,3], une spécificité s’élevant à 90,6 % [95 % IC 87,6 à 93,0] et un rapport de vraisemblance positif de 4,5. La hausse médiane [étendue interquartile] de créatinine sérique par rapport aux valeurs de base était de 16 [-3 à 41] μmol/L pour les patients non codés et de 104 [57 à 158] μmol/L pour les patients chez qui on décelait le code ICD-10-CA [test de Mann-Whitney, p < 0,000 1].

Limite de l’étude

Dans certains cas, la faible sensibilité du code ICD-10-CA pourrait s’expliquer par un code de rejet aigu. Les causes d’épisodes d’IRA n’ont pas pu être établies dans cette étude.

Conclusions

À l’instar de l’ensemble de la population, le code ICD-10 N17x sous-estime l’incidence de l’IRA chez les greffés rénaux lors d’une hospitalisation. Ceci rend l’utilisation de ce code peu propice à des fins d’études sur l’incidence et les conséquences de l’IRA chez les patients hospitalisés qui ont subi une greffe du rein.

What was known before

Validation studies performed in the general population demonstrate that acute kidney injury (AKI) diagnostic codes have low sensitivity but high specificity. The accuracy of AKI diagnostic codes in the kidney transplant population has not been studied.

What this adds

The ICD-10 N17x diagnostic code for AKI has a low sensitivity in the kidney transplant population, making the code unusable for studying the incidence and consequences of AKI in hospitalized kidney transplant patients. These results help inform the conduct of future studies in the kidney transplant population utilizing administrative data.

Background

Health administrative databases house an enormous amount of data that might permit the conduct of large observational studies in an efficient, relatively low-cost manner [1, 2]. However, researchers utilizing such databases must be aware of the limitations of the data and the potential for biased results [3–5]. In particular, the validity of studies for which key exposures or outcomes are identified with diagnostic or procedural codes depends upon the accuracy of such codes [5, 6]. When used for clinical research, the accuracy of diagnostic and procedural codes for the entities they are supposed to represent should be determined.

The accuracy of diagnostic codes for acute kidney injury (AKI) has been measured in the general population, showing a very low sensitivity (approximately 30 %) but a notably high specificity (generally >95 %) [7]. A recent study examined the incidence and outcomes associated with AKI in the kidney transplant population, with AKI being defined using diagnostic codes [8]. However, the accuracy of diagnostic codes for AKI has never been determined in the kidney transplant population.

This study measured the accuracy of the International Classification of Diseases, Tenth Revision (ICD-10) code N17x for AKI in kidney transplant recipients admitted to hospital. We hypothesized that the ICD-10 code would more accurately identify AKI in the kidney transplant population compared to the general population because kidney transplant patients have a higher prevalence of AKI, are more likely to have their kidney function followed closely, and are more likely to have a nephrologist involved in their care during a hospital admission [8–10].

Methods

Study design and setting

We conducted a population-based retrospective validation study in the province of Ontario, Canada, using Ontario’s population-based health administrative databases at the Institute for Clinical Evaluative Sciences (ICES) and laboratory data from Southwestern Ontario and Ottawa, Ontario. Residents of Ontario have universal access to hospital care and physician services under a single provincial payer system, resulting in a comprehensive repository of health administrative data. The availability of laboratory data was limited to Southwestern Ontario and Ottawa, Ontario. The study was conducted according to a pre-specified protocol that was approved by the Ottawa Hospital Research Ethics Board. The reporting of this study follows guidelines set out for studies assessing diagnostic accuracy (Appendix 1) [11].

Data sources

We created our study’s analytical dataset using seven databases that were linked using encrypted unique identifiers. We identified kidney transplant recipients using the Canadian Organ Replacement Register (CORR), which captures data on every kidney transplant in the province of Ontario [12]. Laboratory data were obtained from the Ottawa Hospital Data Warehouse (OHDW) for Ottawa patients and from Cerner and Gamma-Dynacare for Southwestern Ontario patients. OHDW houses inpatient and outpatient lab information for individuals who had bloodwork drawn at one of three hospitals in Ottawa, Ontario. Cerner is a hospital network in Southwestern Ontario, housing inpatient and outpatient laboratory data from 12 hospitals. Gamma-Dynacare is a laboratory service provider that contains outpatient lab information for individuals who had bloodwork drawn at any of their 148 collection sites in Ontario. Demographics and vital status information were obtained from the Ontario Registered Persons Database (which records the sex, birthdate, and death date of all Ontarians) and CORR. Diagnostic and procedural information from all hospitalizations were determined using the Canadian Institute for Health Information Discharge Abstract Database (CIHI-DAD), which captures data on every hospitalization in the province of Ontario. Information was also obtained from the Ontario Health Insurance Plan database, which contains all health claims for inpatient and outpatient physician services. We have previously used these databases to research kidney health outcomes and health services [13–15].

Study cohort

We included patients with the following characteristics: (a) first kidney-only transplant recipients; (b) hospitalized 6 months or later following kidney transplantation; (c) having at least one serum creatinine value measured during the hospital admission; (d) discharged from hospital prior to the end date of laboratory data availability; and (e) serum creatinine data available anytime between 2 weeks to 6 months prior to the admission date to determine baseline creatinine. Hospital admissions less than 6 months post transplant were excluded to eliminate as much as possible AKI secondary to post-operative complications, delayed graft function, and early acute rejection. Hospital admissions occurring between April 1, 2003 and December 31, 2012 (Ottawa) and March 31, 2012 (Southwestern Ontario) were eligible for inclusion. Hospital admissions with an admission date prior to April 1st, 2003 were excluded due to the use of ICD-9 diagnostic codes prior to this date. Originally, ICD-9 codes were included as a separate analysis, but this analysis had to be suppressed (as per ICES privacy regulations), due to the presence of too many small cells (total n = 118 patients). A look-back period of 3 years from the date of hospital admission was used to determine comorbidities. Codes used to define comorbidities of interest are outlined in Appendix 2. When multiple eligible hospital admissions were available for a patient, one was selected at random in order to avoid clustering in the analysis.

Criteria for AKI

We used the Acute Kidney Injury Network (AKIN) staging system to define AKI [16]. AKIN stage 1 is defined by an increase in serum creatinine ≥26.4 μmol/L (0.3 mg/dL) or a 1.5- to 2-fold increase from baseline. AKIN stage 2 is defined by a >2- to 3-fold increase in serum creatinine from baseline. AKIN stage 3 is defined by an increase in serum creatinine >3-fold from baseline, or a serum creainine >354 μmol/L, with an acute increase of at least 44 μmol/L (0.5 mg/dL) [16]. The urine output criterion for the AKIN staging system was not used as these data were not available from our databases. The peak creatinine during a hospital admission was used to define the presence or absence of AKI and the AKIN stage. If multiple baseline creatinine values were available, the most recent value was used, except if drawn less than 2 weeks prior to admission. Creatinine values drawn very close to admission were excluded due to a heightened chance of the patient being unwell at the time of the bloodwork; the result may therefore not reflect a true baseline value but possibly the beginning of the AKI episode.

ICD-10 code for AKI

Trained coders review all charts to record appropriate diagnosis codes and their associated attributes following a discharge from hospital. Coders follow the Canadian Coding Standards developed by CIHI [17]. According to CIHI’s guidelines, the coders are not permitted to interpret laboratory tests; however, they can record a condition based on laboratory measurements if a physician documents the condition in the patient’s chart. For hospitalization records (included in the CIHI-DAD), coders may record up to 25 conditions using ICD-10 diagnostic codes. They must also indicate the diagnosis type. A diagnosis type “M” is the main or most responsible diagnosis, which is the condition that contributed most to the hospital length of stay or used the greatest amount of resources. An admission diagnosis is any condition that existed prior to the admission and was treated during the hospital stay [17].

In our study, we tested the accuracy of the ICD-10 code N17x, which defines “acute renal failure,” when present as diagnosis type “M” or most responsible/main diagnosis, admission type diagnosis, or any diagnosis type (present in any one of the 25 potential diagnosis fields).

Statistical analysis

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value of the ICD-10 N17x code compared to a reference standard of changes in serum creatinine using the AKIN staging system for AKI (formulas and a sample 2 × 2 table are presented in Appendix 3). We calculated 95 % confidence intervals for single proportions using the Wilson Score method [18]. We calculated the positive likelihood ratio using the sensitivity and specificity (Appendix 3). We also compared the change in serum creatinine between patients who were coded positive or negative for N17x. The difference in the distribution of change in creatinine between code negative and code positive patients was formally tested using the Mann-Whitney test. We conducted all analyses using the SAS software, version 9.4 (SAS institute Inc., Cary, NC, USA).

Results

We identified a total of 524 kidney transplant patients with eligible hospital admissions from 2003 to 2012 that met our inclusion criteria. Patient selection is outlined in Additional file 1: Figure S1. Baseline characteristics are outlined in Table 1. The mean (standard deviation, SD) age was 57.7 (12.1) years. The median (interquartile range, IQR) time from kidney transplant to the index hospital admission was 3.5 (1.5, 7.1) years. The baseline serum creatinine was measured a median (IQR) of 34 (22, 68) days prior to hospital admission. The peak creatinine was measured a median (IQR) of 1 (0, 2) day post admission. AKI (based on AKIN criteria) occurred in 45.0 % of the cohort, and 14.1 % of the cohort was coded with ICD-10 N17x. Of the patients with AKI, most (67.8 %) had mild disease (AKIN stage 1).
Table 1

Baseline characteristics

Total n = 524

 

Demographics

 

 Mean age (SD), years

57.7 (12.1)

Age (n (%))

 

 18 to <35

23 (4.4)

 35 to <60

262 (50)

 60 to <70

151 (28.8)

 ≥70

88 (16.8)

Women (n (%))

185 (35.3)

Race (n (%))

 

 Caucasian

374 (71.4)

 Asian

15 (2.9)

 Black

19 (3.6)

 Indian subcontinent

12 (2.3)

 Aboriginal

9 (1.7)

 Other

16 (3.1)

 Unknown

34 (6.5)

 Missing

45 (8.6)

Income quintile (n (%))

 

 One (lowest)

114 (21.8)

 Two

106 (20.2)

 Three

102 (19.5)

 Four

83 (15.8)

 Five (highest)

119 (22.7)

Rural location (n (%))

102 (19.5)

Year of cohort entry (n (%))

 

 2003–2006

149 (28.4)

 2007–2009

159 (30.4)

 2010–2012

216 (41.2)

Time since transplant, median (IQR), years

3.5 (1.5, 7.1)

Year of kidney transplant (n (%))

 

 1988–1992

18 (3.4)

 1993–1997

39 (7.45)

 1998–2002

147 (28.1)

 2003–2007

205 (39.1)

 2008–2012

115 (21.9)

Time on dialysis prior to transplant, median (IQR), years

2.0 (0.9, 3.5)

Time on dialysis prior to transplant (n (%)), years

 

 0 to <1

121 (23.1)

 1 to <2

118 (22.5)

 2 to <3

82 (15.7)

 3 to <4

57 (10.9)

 ≥4

98 (18.7)

Cause of end stage renal disease (n (%))

 

 Glomerulonephritis

148 (28.2)

 Hypertension

49 (9.4)

 Pyelonephritis/interstitial

35 (6.7)

 Cystic kidney disease

61 (11.6)

 Diabetes

108 (20.6)

 Other

29 (5.5)

 Unknown

94 (17.9)

Type of donor (n (%))

 

 Deceased

347 (66.2)

 Living

165 (31.5)

 Missing

12 (2.3)

Comorbidities (n (%))

 

 Coronary artery disease

156 (29.8)

 Diabetes

243 (46.4)

 Hypertension

489 (93.3)

 Congestive heart failure

88 (16.8)

 Chronic liver disease

36 (6.9)

 Chronic obstructive pulmonary disease

20 (3.8)

 Peripheral vascular disease

28 (5.3)

 Stroke/transient ischemic attack

12 (2.3)

Baseline laboratory measurements

 

 Serum creatinine, median (IQR), umol/L

133.0 (103.0, 173.9)

 eGFR, median (IQR), ml/min/1.73 m2a

46.1 (33.4, 62.8)

eGFR category (n (%)), ml/min/1.73 m2

 

 <15

21 (4.0)

 15 to <30

84 (16.0)

 30 to <45

146 (27.9)

 45 to <60

116 (22.1)

 ≥60

157 (30.0)

AKI definitions (n (%))b

 

 Any AKI

236 (45.0)

 AKIN stage 1

160 (30.5)

 AKIN stage 2

21 (4.0)

 AKIN stage 3

55 (10.5)

aEstimated glomerular filtration rate (eGFR) was calculated using the CKD-EPI equation [32]

bDefined by serum creatinine

The diagnostic performance of the various coding algorithms is presented in Table 2. The diagnostic code type of “all diagnoses” performed the best. Compared to a reference standard of AKIN stage 1 or greater, the ICD-10 N17x code for AKI showed a sensitivity of 28.0 % (95 % CI 22.6, 34.0) and specificity of 97.2 % (95 % CI 94.6, 98.6). Compared to a reference standard of AKIN stage 2 or greater, the ICD-10 code showed a sensitivity of 42.1 % (95 % CI 31.7, 53.3) and a specificity of 90.6 % (95 % CI 87.6, 93.0). Overall, specificity was high, >90 % for most code types and definitions of AKI. The positive predictive value decreased significantly with increasing severity of AKI: AKIN stage 1 or greater 89.2 % (95 % CI 80.1, 94.4); AKIN stage 2 or greater 43.2 % (95 % CI 32.6, 54.6); and AKIN stage 3 29.7 % (95 % CI 20.5, 40.9) (Table 2).
Table 2

Diagnostic performance characteristics of three different algorithms for ICD-10 code N17x using the AKIN staging system for AKI as the reference standard

Diagnostic coding algorithm

AKIN stage

Diagnostic performance characteristics (95 % CI)

All diagnoses

AKIN stage 1 or greater

Sn = 28.0 (22.6, 34.0)a

Sp = 97.2 (94.6, 98.6)

PPV = 89.2 (80.1, 94.4)

NPV = 62.2 (57.7, 66.6)

LR+ = 10.0

AKIN stage 2 or greater

Sn = 42.1 (31.7, 53.3)

Sp = 90.6 (87.6, 93.0)

PPV = 43.2 (32.6, 54.6)

NPV = 90.2 (87.1, 92.6)

LR+ = 4.5

AKIN stage 3

Sn = 40.0 (28.1, 53.2)

Sp = 88.9 (85.8, 91.4)

PPV = 29.7 (20.5, 40.9)

NPV = 92.7 (89.9, 94.7)

LR+ = 3.6

Admission diagnosis

AKIN stage 1 or greater

Sn = 20.3 (15.7, 25.9)

Sp = 97.9 (95.5, 99.0)

PPV = 88.9 (77.8, 94.8)

NPV = 60.0 (55.5, 64.3)

LR+ = 9.7

AKIN stage 2 or greater

Sn = 34.2 (24.5, 45.4)

Sp = 93.8 (91.1, 95.6)

PPV = 48.2 (35.4, 61.2)

NPV = 89.4 (86.3, 91.8)

LR+ = 5.5

AKIN stage 3

Sn = 32.7 (21.8, 45.9)

Sp = 92.3 (89.6, 94.4)

PPV = 33.3 (22.2, 46.6)

NPV = 92.1 (89.3, 94.2)

LR+ = 4.2

Main diagnosis/most responsible diagnosis

AKIN stage 1 or greaterb

 

AKIN stage 2 or greater

Sn = 17.1 (10.3, 27.1)

Sp = 97.3 (95.4, 98.5)

PPV = 52.0 (33.5, 70.0)

NPV = 87.4 (84.2, 90.0)

LR+ = 6.3

AKIN stage 3

Sn = 18.2 (10.2, 30.3)

Sp = 96.8 (94.8, 98.1)

PPV = 40.0 (23.4, 59.3)

NPV = 91.0 (88.2, 93.2)

LR+ = 5.7

aUsed the Wilson 95 % confidence interval

bResults suppressed as per ICES privacy regulations due to the presence of small cells

Sn sensitivity, Sp specificity, PPV positive predictive value, NPV negative predictive value, LR+ positive likelihood ratio

The absolute and relative changes in serum creatinine for patients that were coded positive and negative for AKI are presented in Table 3 and Additional file 1: Figures S2 and 3. In patients who were code positive and code negative for AKI, the median (IQR) absolute rise in serum creatinine from baseline was 104 (57 to 158) μmol/L and 16 (−3 to 41) μmol/L, respectively. The median (IQR) percent relative change was 56.9 (35 to 111) and 12.9 (−2.2 to 31), for code positive and code negative patients, respectively. The difference in the distribution of the absolute and relative changes in serum creatinine between code negative and code positive patients was statistically significant when the distributions were compared using the Mann-Whitney test (p < 0.0001).
Table 3

Change in serum creatinine from baseline in all patients with and without an ICD-10 N17x code for AKI (referred to as code positive or code negative)

Diagnostic coding algorithm

Code

Number

Absolute change (μmol/L)

Median (IQR)

Relative change (%)a

All diagnoses

+

74

104.2 (57.0 to 158.0)

56.9 (35.0 to 111.4)

−

450

16.0 (−3.0 to 41.0)

12.9 (−2.2 to 30.5)

Admission diagnosis

+

54

109.0 (62.0 to 161.0)

64.0 (35.0 to 122.1)

−

470

18.0 (−2.0 to 45.0)

13.8 (−1.5 to 32.5)

Main diagnosis/most responsible diagnosis

+

25

128.0 (57.0 to 160.0)

71.3 (27.4 to 122.1)

−

499

19.0 (−1.0 to 51.0)

15.3 (−0.7 to 37.4)

aRelative change = (peak serum creatinine − baseline serum creatinine)/baseline serum creatinine × 100

Discussion

This study measured the accuracy of the ICD-10 N17x code for the diagnosis of AKI in the kidney transplant population. The best performing code type was “all diagnoses” (i.e., present in any diagnostic field during a hospital admission). All code types, when compared to all definitions of AKI, showed a low to moderate sensitivity but high specificity. For AKI AKIN stage 1 or greater (any type of AKI), the positive predictive value of the code was high at almost 90 %. This suggests that the code would be reasonable for cohort selection if only kidney transplant patients with AKI were of interest. However, one would have to be satisfied that there was no differential misclassification, given the low sensitivity of the code, resulting in the exclusion of many patients who truly have AKI. Also, if only patients with more severe forms of AKI (AKIN stage 2 or 3) were of interest, the code would not be appropriate for cohort creation given the low prevalence of this disease stage with a resultant low positive predictive value.

The code performed poorly when a code type of “main or most responsible diagnosis” was used. This could be due to the fact that AKI often occurs in the setting of another illness, such as an infection [9, 19], which may be coded as the main diagnosis as opposed to AKI. The positive predictive value of the code was quite variable depending on the reference standard used. A low positive predictive value for severe AKI (AKIN stage 3) was found with all code types. This is likely due to the very low prevalence of AKIN stage 3 (10.5 %) in our cohort. The positive predictive value of a test (in this case the code) is known to vary significantly depending on the prevalence of the disease [20]. Specificity was quite high for all code types; however, it was slightly lower for code types with a higher sensitivity. Most code types were more sensitive (less false negatives) when a higher stage of AKI was used as the reference standard. This is to be expected because severe AKI is more clinically apparent and therefore more likely to be recorded in the chart.

The sensitivity of the code was lower than expected. When examined in healthy elderly and elderly patients with chronic kidney disease (CKD) (both at higher risk for AKI [21–23], similar to kidney transplant patients), the ICD-10 code for AKI had a sensitivity of 62 and 76 % respectively, compared to a reference of at least a doubling in serum creatinine [24]. One possible reason for the lower than expected sensitivity of the code is that acute rejection (a cause of AKI), has its own diagnostic code. Although we could not verify the true incidence of acute rejection in our study, it should be very low given that the median time from transplant to index admission was 3.5 years, and acute rejection is very infrequent beyond the first year [25]. Nonetheless, a small proportion of AKI episodes (determined based on the reference standard of a rise in creatinine) were likely assigned a code for acute rejection as opposed to AKI. The specificity of the code in the kidney transplant population was slightly lower than in the general or elderly population [7], suggesting that kidney transplant patients are more likely to have a code assigned for AKI when their kidney function is actually stable. Overall, the code demonstrated limited sensitivity; however, a high positive predictive value for any AKI was found. The code was also able to distinguish two populations with significantly different rises in serum creatinine.

To our knowledge, this is the first study to measure the accuracy of the ICD-10 diagnostic code for AKI in the kidney transplant population. Prior studies have examined the accuracy of AKI codes, mostly ICD-9, in the general, elderly, and elderly CKD populations [7, 24, 26]. We studied transplant patients from two healthcare regions in the province of Ontario, making the sample more representative and thus generalizable. We had serum creatinine values available, making it possible to compare the administrative diagnostic code to the gold standard for diagnosing AKI, as opposed to relying on chart re-abstraction.

Limitations to our study should be noted. First, for the reference standard used to define AKI, we adapted the creatinine-based component of the AKIN classification system, which defines AKI using both serum creatinine and urine output measurements [16]. It is also recommended that the AKIN classification system be applied only after a patient has achieved an optimal state of hydration. Unfortunately, urine output measurements and clinical data, such as hydration status and the administration of intravenous fluids, were not available in the administrative datasets that we used for this study. However, even if urine output data were available, urine output measurements are difficult to obtain and are poorly documented outside the intensive care setting. In addition, the sole use of serum creatinine is a commonly accepted method of defining AKI, both clinically and for research purposes [9, 27, 28]. Second, there is no consensus definition for AKI that has been validated in the kidney transplant population [29, 30]. However, all established classification systems apply similar serum creatinine and urine output criteria [30, 31]; AKI is defined similarly for transplant and non-transplant patients in the clinical setting; and the AKIN staging system was used to define AKI and correlated with poor outcomes in a prior study of kidney transplant patients [9]. Third, we did not specify a timeline of <48 h within which AKI had to occur (as specified in the AKIN criteria). The use of stringent timelines would likely diminish the accuracy of the AKI code since these timelines are not applied in the clinical setting. Finally, data on the cause of AKI was not available. The accuracy of administrative coding for AKI may differ depending on the cause, especially in transplantation, where a diagnosis of acute rejection may be coded preferentially over a diagnosis of AKI.

Conclusions

In summary, our study demonstrates that identifying AKI in kidney transplant patients using administrative diagnostic codes will result in an underestimation of the true incidence and misclassification of patients with AKI. This limitation makes the code ineffectual for determining the incidence and consequences of AKI in hospitalized kidney transplant patients.

Abbreviations

AKI: 

acute kidney injury

AKIN: 

acute kidney injury network

CIHI: 

Canadian Institute for Health Information

CKD: 

chronic kidney disease

ICD-10: 

International Classification of Diseases, Tenth Revision

ICES: 

Institute for Clinical Evaluative Sciences

PPV: 

positive predictive value

Declarations

Acknowledgements

This study was conducted at the Institute for Clinical Evaluative Sciences (ICES) Western Site. ICES is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care. Core funding for ICES Western is provided by the Academic Medical Organization of Southwestern Ontario (AMOSO), the Schulich School of Medicine and Dentistry (SSMD), Western University, and the Lawson Health Research Institute (LHRI). This project was conducted with members of the provincial ICES Kidney, Dialysis and Transplantation Research Program (www.ices.on.ca/kdt), which receives programmatic grant funding from the Canadian Institutes of Health Research. Parts of this material are based on data and information compiled and provided by CIHI. The opinions, results, and conclusions are those of the authors and are independent from the funding sources. No endorsement by ICES, AMOSO, SSMD, LHRI, CIHI, or the MOHLTC is intended or should be inferred. Dr. Amit Garg is supported by the Dr. Adam Linton Chair in Kidney Health Analytics. Dr. Greg Knoll is supported by the University of Ottawa Chair in Clinical Transplantation Research. Cr. Carl van Walraven is supported by the University of Ottawa Department of Medicine. Research personnel who worked on this project were supported by the Lilibeth Caberto Kidney Clinical Research Unit. Dr. Amber Molnar received salary support from the KRESCENT Foundation and the University of Ottawa, Department of Medicine.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Division of Nephrology, Department of Medicine, McMaster University
(2)
Clinical Epidemiology Program, Ottawa Hospital Research Institute
(3)
Institute for Clinical Evaluative Sciences
(4)
Department of Medicine, University of Ottawa
(5)
Department of Epidemiology & Biostatistics, Western University
(6)
Division of Nephrology, Western University
(7)
Division of Nephrology, Department of Medicine, University of Ottawa
(8)
St. Joseph’s Healthcare

References

  1. Iezzoni LI. Assessing quality using administrative data. Ann Intern Med. 1997;127(8 Pt 2):666–74.View ArticlePubMedGoogle Scholar
  2. Mohammed MA, Stevens A. The value of administrative databases. BMJ. 2007;334(7602):1014–5. doi:https://doi.org/10.1136/bmj.39211.453275.80.View ArticlePubMedPubMed CentralGoogle Scholar
  3. Grimes DA. Epidemiologic research using administrative databases: garbage in, garbage out. Obstet Gynecol. 2010;116(5):1018–9. doi:https://doi.org/10.1097/AOG.0b013e3181f98300.View ArticlePubMedGoogle Scholar
  4. Grimes DA. Epidemiologic research with administrative databases: red herrings, false alarms and pseudo-epidemics. Hum Reprod. 2015. doi:https://doi.org/10.1093/humrep/dev151.PubMedGoogle Scholar
  5. van Walraven C, Austin P. Administrative database research has unique characteristics that can risk biased results. J Clin Epidemiol. 2012;65(2):126–31. doi:https://doi.org/10.1016/j.jclinepi.2011.08.002.View ArticlePubMedGoogle Scholar
  6. van Walraven C, Bennett C, Forster AJ. Administrative database research infrequently used validated diagnostic or procedural codes. J Clin Epidemiol. 2011;64(10):1054–9. doi:https://doi.org/10.1016/j.jclinepi.2011.01.001.View ArticlePubMedGoogle Scholar
  7. Vlasschaert ME, Bejaimal SA, Hackam DG, Quinn R, Cuerden MS, Oliver MJ, et al. Validity of administrative database coding for kidney disease: a systematic review. Am J Kidney Dis. 2011;57(1):29–43. doi:https://doi.org/10.1053/j.ajkd.2010.08.031.View ArticlePubMedGoogle Scholar
  8. Mehrotra A, Rose C, Pannu N, Gill J, Tonelli M, Gill JS. Incidence and consequences of acute kidney injury in kidney transplant recipients. Am J Kidney Dis. 2012;59(4):558–65. doi:https://doi.org/10.1053/j.ajkd.2011.11.034.View ArticlePubMedGoogle Scholar
  9. Nakamura M, Seki G, Iwadoh K, Nakajima I, Fuchinoue S, Fujita T, et al. Acute kidney injury as defined by the RIFLE criteria is a risk factor for kidney transplant graft failure. Clin Transplant. 2012;26(4):520–8. doi:https://doi.org/10.1111/j.1399-0012.2011.01546.x.View ArticlePubMedGoogle Scholar
  10. Nehus EJ, Devarajan P. Acute kidney injury: AKI in kidney transplant recipients—here to stay. Nat Rev Nephrol. 2012;8(4):198–9. doi:https://doi.org/10.1038/nrneph.2012.40.View ArticlePubMedGoogle Scholar
  11. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ. 2003;326(7379):41–4.View ArticlePubMedPubMed CentralGoogle Scholar
  12. Lam NN, McArthur E, Kim SJ, Knoll GA. Validation of kidney transplantation using administrative data. Can J Kidney Health Dis. 2015;2:20. doi:https://doi.org/10.1186/s40697-015-0054-9.View ArticlePubMedPubMed CentralGoogle Scholar
  13. Molnar AO, Coca SG, Devereaux PJ, Jain AK, Kitchlu A, Luo J, et al. Statin use associates with a lower incidence of acute kidney injury after major elective surgery. J Am Soc Nephrol. 2011;22(5):939–46. doi:https://doi.org/10.1681/ASN.2010050442.View ArticlePubMedPubMed CentralGoogle Scholar
  14. Naylor KL, McArthur E, Leslie WD, Fraser LA, Jamal SA, Cadarette SM, et al. The three-year incidence of fracture in chronic kidney disease. Kidney Int. 2014;86(4):810–8. doi:https://doi.org/10.1038/ki.2013.547.View ArticlePubMedGoogle Scholar
  15. Sood MM, Bota SE, McArthur E, Kapral MK, Tangri N, Knoll G, et al. The three-year incidence of major hemorrhage among older adults initiating chronic dialysis. Can J Kidney Health Dis. 2014;1(21):1–10.PubMedPubMed CentralGoogle Scholar
  16. Mehta RL, Kellum JA, Shah SV, Molitoris BA, Ronco C, Warnock DG, et al. Acute Kidney Injury Network: report of an initiative to improve outcomes in acute kidney injury. Crit Care. 2007;11(2):R31. doi:https://doi.org/10.1186/cc5713.View ArticlePubMedPubMed CentralGoogle Scholar
  17. Canadian coding standards for version 2012 ICD-10-CA and CCI [database on the Internet]. Available from: https://secure.cihi.ca/free_products/canadian_coding_standards_2012_e.pdf. Accessed: 10 July 2015.
  18. Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998;17(8):857–72.View ArticlePubMedGoogle Scholar
  19. Bardak S, Turgutalp K, Turkegun M, Demir S, Kiykim A. Recurrent acute kidney injury in renal transplant patients: a single-center study. Transplant Proc. 2015;47(5):1437–41. doi:https://doi.org/10.1016/j.transproceed.2015.04.077.View ArticlePubMedGoogle Scholar
  20. Society, the individual, and medicine [database on the Internet]. Available from: http://www.med.uottawa.ca/sim/data/Sensitivity_and_Prevalence_e.htm. Accessed: 16 July 2015.
  21. Hsu CY, McCulloch CE, Fan D, Ordonez JD, Chertow GM, Go AS. Community-based incidence of acute renal failure. Kidney Int. 2007;72(2):208–12. doi:https://doi.org/10.1038/sj.ki.5002297.View ArticlePubMedPubMed CentralGoogle Scholar
  22. Hsu CY, Ordonez JD, Chertow GM, Fan D, McCulloch CE, Go AS. The risk of acute renal failure in patients with chronic kidney disease. Kidney Int. 2008;74(1):101–7. doi:https://doi.org/10.1038/ki.2008.107.View ArticlePubMedPubMed CentralGoogle Scholar
  23. Xue JL, Daniels F, Star RA, Kimmel PL, Eggers PW, Molitoris BA, et al. Incidence and mortality of acute renal failure in Medicare beneficiaries, 1992 to 2001. J Am Soc Nephrol. 2006;17(4):1135–42. doi:https://doi.org/10.1681/ASN.2005060668.View ArticlePubMedGoogle Scholar
  24. Hwang YJ, Shariff SZ, Gandhi S, Wald R, Clark E, Fleet JL et al. Validity of the International Classification of Diseases, Tenth Revision code for acute kidney injury in elderly patients at presentation to the emergency department and at hospital admission. BMJ open. 2012;2(6). doi:https://doi.org/10.1136/bmjopen-2012-001821.
  25. Cooper JE, Wiseman AC. Acute kidney injury in kidney transplantation. Curr Opin Nephrol Hypertens. 2013;22(6):698–703. doi:https://doi.org/10.1097/MNH.0b013e328365b388.View ArticlePubMedGoogle Scholar
  26. Grams ME, Waikar SS, MacMahon B, Whelton S, Ballew SH, Coresh J. Performance and limitations of administrative data in the identification of AKI. Clin J Am Soc Nephrol. 2014;9(4):682–9. doi:https://doi.org/10.2215/CJN.07650713.View ArticlePubMedPubMed CentralGoogle Scholar
  27. Molnar AO, Parikh CR, Coca SG, Thiessen-Philbrook H, Koyner JL, Shlipak MG, et al. Association between preoperative statin use and acute kidney injury biomarkers in cardiac surgical procedures. Ann Thorac Surg. 2014;97(6):2081–7. doi:https://doi.org/10.1016/j.athoracsur.2014.02.033.View ArticlePubMedPubMed CentralGoogle Scholar
  28. Zhang WR, Garg AX, Coca SG, Devereaux PJ, Eikelboom J, Kavsak P, et al. Plasma IL-6 and IL-10 concentrations predict AKI and long-term mortality in adults after cardiac surgery. J Am Soc Nephrol. 2015. doi:https://doi.org/10.1681/ASN.2014080764.Google Scholar
  29. Cruz DN, Bagshaw SM, Ronco C, Ricci Z. Acute kidney injury: classification and staging. Contrib Nephrol. 2010;164:24–32. doi:https://doi.org/10.1159/000313717.View ArticlePubMedGoogle Scholar
  30. Joannidis M, Metnitz B, Bauer P, Schusterschitz N, Moreno R, Druml W, et al. Acute kidney injury in critically ill patients classified by AKIN versus RIFLE using the SAPS 3 database. Intensive Care Med. 2009;35(10):1692–702. doi:https://doi.org/10.1007/s00134-009-1530-4.View ArticlePubMedGoogle Scholar
  31. Khwaja A. KDIGO clinical practice guidelines for acute kidney injury. Nephron Clin Pract. 2012;120(4):c179–84. doi:https://doi.org/10.1159/000339789.View ArticlePubMedGoogle Scholar
  32. Levey AS, Stevens LA, Schmid CH, Zhang YL, Castro 3rd AF, Feldman HI, et al. A new equation to estimate glomerular filtration rate. Ann Intern Med. 2009;150(9):604–12.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© Molnar et al. 2016

Advertisement