Foramen Ovale, Patent

Studies have shown that ICD-9-CM codes for CHDs lack specificity and have poor diagnostic accuracy. (Cronk et al., 2003 (link); Frohnert et al., 2005 (link); Strickland et al., 2008 (link); Rodriguez et al., 2018 (link)) For example, 745.5 codes for both atrial septal defect (ASD), a true CHD, and patent foramen ovale (PFO), a normal newborn condition. The PFO, which may persist in 25% of adults, is usually clinically insignificant and asymptomatic, but potentially associated with strokes. The 745.4 codes for ventricular septal defect, a common congenital condition which may also be acquired in adults after a myocardial infarction. To investigate the validity of specific codes for identifying a true CHD, clinicians at participating sites reviewed medical records from a random sample of cases. Each site had different sample size and code selection (Table 2) based on number of cases with codes of interest, availability of medical records, and clinical review resources. Cases sampled had the following isolated codes potentially used for non-CHD conditions: 745.5, 745.4 in persons >40 years old, 746.85 (coronary artery anomaly), and 746.89/746.9 (other specified/unspecified heart anomaly).

We used data from the Stanford Translational Research Integrated Database Environment (STRIDE) which contains clinical information of over 2 million pediatric and adult patients cared for at Stanford Health Care and Stanford Children’s Health from 1995 to 2015, including 20 million patient encounters with transcriptions of all inpatient and outpatient clinical notes, pathology and radiology reports, medication lists, lab results, and vitals data. This data source was accessed under approved Institutional Review Board protocols.
Through a previously validated and implemented text-processing pipeline to analyze clinical data [8 (link)–10 (link)], we used Unitex [11 ] as an annotator and over 10 clinical ontologies to extract positive present mentions of disease concepts from all clinical notes. We excluded uninformative phrases based on the term frequency analysis [12 (link)] and kept only terms with more than 4 characters to avoid ambiguity. We also flagged negative mentions (e.g. “ruled out stroke”) and determined if a term was from patients’ history or family history sections of a note [13 (link)]. The product of this pipeline is a list of present, positive mentions of biomedical concepts in each patient note.
We identified all patients who had their first ICD-9 documentation of CS/TIA at age 40 or older in either inpatient and outpatient encounters. The inclusion criteria using ICD-9 diagnosis codes are stroke (434 and 436) and TIA (435.9). These ICD-9 codes were selected because have been previously shown to have high specificity and sensitivity for ischemic stroke when confirmed with chart review[14 ].
From the CS/TIA patients identified using these codes, some were removed from the cohort based on both ICD-9 and clinical text evidence that meets specific exclusion criteria to increase specificity for patients without these conditions. Patients who had ICD-9 diagnosis of carotid artery occlusion or stenosis (433.1), intracranial hemorrhage (431), and atrial septal defects (745.5) were excluded as these are identifiable etiologies of stroke. Patients with rheumatic heart disease (433.1) or prosthetic valve(s) (V43.3) were excluded as AF in these contexts belong to a separate entity, valvular AF, separate from AF of the general population. Those with hyperthyroid disease (242.9) were also excluded as this is a known reversible cause of AF. Patients who had clinical text evidence of rheumatic heart disease, prosthetic valve(s), and/or patent foramen ovale were also excluded.
The outcome of interest in this study is diagnosis of AF after CS/TIA. All patients who had history of AF were identified by ICD-9 code (427.31 and 427.32). Those positive for AF were defined as patients over 40 years old with CS/TIA whose first ICD-9 documentation of AF was at least 30 days after first episode of CS/TIA. We used a 30-day cutoff to exclude patients who may have had delayed documentation of AF related to their hospitalization for initial stroke. Those negative for AF were defined as patients with CS/TIA with no ICD-9 documentation of AF during the extent of their follow-up as documented in their records.
Basic demographic information such as age at time of CS/TIA and sex were obtained from the structured fields of their records. Risk factors were extracted based on ICD-9 documentation at any time point of patients’ records to enable us to better capture those patients’ chronic conditions. Risk factors assessed were hypertension (HTN), diabetes (DM), obesity defined by BMI>30, systolic and/or diastolic heart failure (CHF), coronary artery disease (CAD), peripheral vascular disease (PVD), chronic kidney disease (CKD) stage III, IV, V, aortic valve disease, mitral valve disease, tricuspid valve disease, and pulmonary valve disease. Such clinical factors are well known to be risk factors for AF [15 (link),16 (link)]. A comprehensive list of conditions covered by each respective ICD-9 used is detailed in the supplemental table.
We randomly split the total cohort into 2 groups: the first for model derivation (80%) and the second for model validation (20%). Candidate predictor variables include age, sex and all the risk factors described above. Univariable logistic regression was first applied to identify the association between each of the predictor variables and diagnosis of AF in the derivation cohort. A multivariable logistic regression model with stepwise variable selection was then trained on the derivation cohort to identify predictors of AF and to estimate their relative predictive power. A simplified risk stratification system was developed based on the beta coefficients of the multivariable logistic regression model as validated by previously published methods [17 (link)]. Points assigned to each significant risk factor were obtained by dividing each by the lowest coefficient and rounding to the nearest integer [18 (link)]. We then calculated a patient’s risk score by summing up all points that correspond to the risk factors present in the given patient’s record.
We assessed model discrimination by using the c-statistic, or area under the curve (AUC) of the Receiver Operating Characteristic (ROC) Curve, which defines how well a model or prediction rule can discriminate between patients who are and are not positive for an event. We then used the Cochran-Armitage trending statistic to assess the ability of the risk score system to differentiate low-risk from high-risk patients. The scoring system was applied to and evaluated in the derivation cohort to assess its applicability. The performance of HAVOC and CHA₂DS₂-Vasc was compared at a score of 4, which was the cutoff value between the low and medium risk strata for both scoring systems. McNemar’s Chi- squared tests were used to compare the sensitivity, specificity, and accuracy. Positive predictive values (PPV) and negative predictive values (NPV) were compared using a test score developed by Leisenring et al. [19 (link)] All analyses were performed using open source statistical program R Version 3.2.2 [20 ].