Evaluating and Improving the Accuracy of ICD-Coded Hospital Data Systems in Estimating the Incidence of Nonfatal Firearm Injuries by Intent Type

Overview

Estimates of nonfatal firearm injury drawn from routinely-collected hospital billing data underestimate assaults and overestimate unintentional injuries. This project will describe the extent to which these discrepancies occur at two study sites (one in Massachusetts, the other in Washington State), identify the underlying reasons for biased estimates, and develop approaches that more accurately classify hospital-treated firearm injuries. A one-year extension project will assess the accuracy of intent coding of trauma registry data.

Status

Complete

Purpose

This project seeks to improve hospital data systems that track the incidence of nonfatal firearm injuries by type of incident (assault, unintentional, legal intervention, and self-inflicted) and, in the extension project, to assess the accuracy of intent coding of trauma registry data.

Approach

We will review 3,000 medical charts to establish a 'gold standard' classification of firearm injuries by intent type. The gold standard will enable us to: 1) identify meaningful biases in routinely coded records (both for hospital-billing data and for trauma registry data) and, after further interrogating the causes of discrepancies, propose practical solutions for current problems with ICD-coding and 2) develop a Natural Language Processing/Artificial Intelligence (NLP/AI) algorithm, applied to electronic medical records, that could be implemented by hospitals to more accurately classify nonfatal firearm injuries seen in their emergency departments. We will share our findings with the committee that sets standards for ICD-coding practices nationwide and make the NLP/AI algorithms we develop publicly available via the Open Science Framework website. The aims of the extension project, beyond documenting the accuracy of the trauma registry coding of intent, include collaborating with the American College of Surgeons to conduct a survey of hundreds of trauma registrars to assess coding practices for a series of hypothetical cases and, afterwards, to conduct interviews with trauma registrars in different parts of the country to provide insight into how trauma centers approach the adjudication of firearm injury intent for registry purposes.

Significance

Our project provides a way to improve current surveillance of hospital-treated firearm injuries by providing empirically developed guidelines that overcome systematic biases in existing data collection processes. It also provides hospitals with an AI tool to efficiently and accurately classify intent to firearm injuries seen in their emergency departments.

Published Research

• Assessment of the Accuracy of Firearm Injury Intent Coding at 3 US Hospitals

  View research

• Classifying Firearm Injury Intent in Electronic Hospital Records Using Natural Language Processing

  View research

• Using Multiple Imputation by Super Learning to Assign Intent to Nonfatal Firearm Injuries

  View research (subscription required) View research on OSF

• Comment: The Problem With ICD-Coded Firearm Injuries

  View letter (subscription required)

Investigator Bios

• Matthew Miller, professor of health sciences and epidemiology at Northeastern University, adjunct professor of Epidemiology at the Harvard Chan School of Public Health, and co-director of the Harvard Injury Control Research Center, is a nationally recognized expert in injury and violence prevention, with expertise if firearm-related violence, self-harm, interpersonal firearm injury, and pharmacoepidemiology using hospital data systems.
• Deborah Azrael, research director at the Harvard Injury Control Research Center, is an expert in firearm-related violence and injury surveillance, and, along with Catherine Barber, was instrumental in developing the National Violent Death Reporting System.
• Catherine Barber, director of the Means Matter campaign at the Harvard Injury Control Research Center, is an expert in injury surveillance and firearm-injury prevention. Along with Deborah Azrael, she was instrumental in developing the National Violent Death Reporting System.
• Eric Goralnick, medical director, Emergency Preparedness and Access Center, Brigham Health, and associate professor, Harvard Medical School, is an expert in firearm and other traumatic injuries.
• Stephen Mooney, assistant profesor of epidemiology, University of Washington.
• Ali Rowhani-Rahbar, Bartley Dobb Associate Professor for the Study and Prevention of Violence and Associate Professor of Epidemiology at the University of Washington, is the Co-Director of the Firearm Injury and Policy Research Program, Leader of the Violence Prevention Section, and Director of the Research Core at the Harborview Injury Prevention and Research Center. Ali brings to the project his methodological and substantive expertise in gun violence research that attends to the antecedents and consequences of intent-specific firearm-related injuries.
• Li Zhou, associate professor of medicine, Division of General Internal Medicine and Primary Care of the Brigham and Women’s Hospital. Dr. Zhou’s primary research areas include bioinformatics, natural language processing (NLP), temporal reasoning, knowledge representation, terminology standards, and clinical decision support.