I'd like to thank the organizers of the ACC for the opportunity to present our experience using automated data abstraction utilizing natural language processing here at Lucio Packard Children's Hospital at Stanford. Some of my disclosures include that I am a medical advisor to CARTA Healthcare and receive philanthropic funding from Susan and Thomas Tobiason Foundation. What I believe is a topic that is quite germane to us all is that we're all a part of clinical registries and we all feel that registries are foundationally important for us to learn and improve our systems of healthcare. It is a systematic way of aggregating and capturing data on patients with similar diseases or conditions or procedures and that we can improve outcomes by aggregating the data and drive collaborative networks of clinical teams. We can utilize the social networking of the registries for patients and families and ultimately because of our participation in registries and learning collaboratives, we can enhance our clinical and academic reputation throughout. Yet when we take a step back, it is quite astonishing the sheer number of registries we participate in. At our hospital, we performed a simple survey to canvas the numbers, types, and importantly the infrastructural support behind all our clinical registries and found more than 70 different and active registries and learning collaboratives across our specialty divisions and departments. We found that labor to support these registries are collectively and perniciously high. The registries are structured in silos with redundancy in data capture and cost structure. When taking a closer look, we surmise that the real rate limiting step is in the fact that most of the cost is associated with manual data abstraction, which is the direct cost to support a set of individuals, usually clinically trained, to manually abstract data from an electronic platform, namely the electronic health record, and data by data enter it into a separate data platform utilized by the registries. The process by all industry standards is inefficient, incomplete, error prone, and poses important restrictions on the amount of data that can be collected with implications on the types of research questions we can ask of registries. Ultimately, we feel this is an unsustainable system that is a real watershed area in our ability to leverage registries to their fullest potential to impact population-based research and quality improvement. This provides the backdrop of why we embarked on a journey to explore how modern industry standards utilizing branches of artificial intelligence may help with the efficiency and effectiveness for hospitals and their registries. One such subset of AI, Natural Language Processing, or NLP, is the overarching term used to describe the process of using computer algorithms to identify key elements in everyday language and extract meaning from unstructured spoken or written input, and is an essential tool for modern day organizations. Modern industries use NLP routinely. Internet search engines, email filters, smart assistants like Alexa or Siri are everyday examples. And even though language doesn't follow a strict set of rules with so many exceptions like I before E except after C, NLP is gaining momentum with greater human dependency on computing systems to communicate and perform tasks. With registries, where the collected patient variables are pre-specified and rigorously defined, the application of NLP to automate chart abstraction is a task that has the probability of good success with meaningful impact to improve efficiency and the whole value proposition for registry participation. What happens to our patient in the hospital are captured through a combination of values that can be categorized as structured data, which are like your laboratory values, your vital signs, intracardiac pressure monitors, and unstructured data, such as free texts within clinical notes or operative notes or even radiographic readings. Unfortunately, contemporary EHR systems are structurally challenged where the abstraction of data, particularly unstructured data, is oftentimes prohibitively difficult. And EHR systems to date do not have the interoperability standards to freely communicate with outside vendor systems that usually house our registry data. NLP is one potential solution to curate both structured and unstructured data that is otherwise locked in our EHR systems into what we call a knowledge cloud for each patient. This knowledge cloud is simply a more user-friendly mathematical representation of a patient from which suggestions to populate registry fields can begin. Our aim with NLP is to improve the efficiency of chart abstraction in order to increase the bandwidth for our nurse abstractors. Over the past three years, we developed the software in partnership with a third-party vendor to determine the impact of NLP on our existing cardiac registries, such as PC4 and IMPACT. Our goal was to improve the efficiency of data abstraction to maintain the recommended case submission rate, something which we were falling badly behind in, demonstrate a lower-cost model, and develop a pipeline where all of the data that we were abstracting can be fed back into our data warehouses for local research and quality improvement opportunities. Early on, we found that NLP-assisted chart abstraction was very good at auto-populating structured data fields that required computing or logic. For example, the vasoactive inotrope score is something that is calculated based on the type of inotrope and the dose and is trended over time. This is just a snippet of the data abstraction form that our nurse abstractor must complete for each patient in our cardiac intensive care unit. For each patient, the vasoactive inotrope score is calculated over several time periods, which for each patient is time-consuming and, quite frankly, mind-numbing. On average, it takes approximately 10 minutes per patient to complete. Now, our typical case rate is approximately 25 patients per week, which is more than four hours per week or 16 hours per month devoted to just one out of more than 400 variables in our registry form. In our early experience, NLP is able to automate the completion of this task nearly instantaneously with substantial savings in time. Another example is utilizing NLP for unstructured data. In the case of inadvertent extubations, which is an up-and-coming healthcare-associated condition adopted by the Solutions for Patient Safety, tracking inadvertent extubations is quite difficult because the adjudication between accidental or intentional extubation is not well characterized by structured data. For a human abstractor, determining the nature of an extubation event requires extensive chart reviews to determine the intent of the extubation. Bringing these pieces of information together is another example where the application of NLP can derive what we call an evidence trail for abstractors to be able to quickly review and adjudicate whether an extubation occurred and whether it was purposeful or accidental. For the Cardiac Registry PC4, we targeted 100 fields representing a combination of structured and unstructured data to determine the feasibility of NLP to assist in the data abstraction through automated suggestions. This was a single-center experience with Epic as the EHR vendor. We were able to map all 100 fields from the EHR, and because of the effort was estimated to be relatively neutral, we were able to pick up an additional 300 fields from the registry form. In all, we estimated that having NLP suggest answers to our nurse abstractors improves their efficiency by about 67% as measured by time series analysis. It's important to note that this technology has since been tested in other EHR vendors and is not unique to the Epic ecosystem. From this and other experiences, we learned that NLP as a tool for registry support is great at some things but struggle with others. NLP is particularly good for registry variables that require computation or logic, requires multiple sources to derive, variables that are documented in discrete or structured fields, or are documented in clinical notes as free texts are particularly amenable to NLP, and those variables that have standardized data definitions which most registries require. However, NLP still struggles with variables that require adjudication from a third party. For example, central line-associated bloodstream infections require adjudication from the hospital's infection control team, which is often not consistently cataloged in a patient's EHR. NLP also struggles with variables that are flexible in interpretation or are erroneously documented without other evidence to contradict. However, these would also be difficult for a human abstractor to contend with, so in essence what is hard for a human to clinically interpret would not be made easier with NLP. Like nearly all instances of artificial intelligence, the optimal workflow is not about human versus computer, but rather how the two can work synergistically with each other to develop a superior result. Up until now, I've talked about the feasibility and the utility of NLP as a technology to improve chart abstraction. What we discovered throughout this three-year journey is that simply improving the user interface between the chart abstractor and the electronic health record utilizing NLP in and of itself improves efficiency. The NLP engine develops an evidence trail for each variable associated with the registry. This evidence trail is readily apparent to the abstractor so that instead of digging around the patient's chart note after note or screen after screen, all the relevant documents are readily made available for the abstractor to adjudicate with greater efficiency. You can also see on the right there is an event timeline for a patient's hospitalization, and each registry variable can be discovered through a relatively easy-to-understand representation of the patient's hospital course. Search functions within the user interface also adds to the ease, enabling abstractors to essentially surf the health record for hard-to-find values. There's obviously more than I could cover than the time allowed, but when we started out on this journey three years ago, we developed a partnership with now the stand-alone third-party vendor Carta Healthcare to improve the efficiency of data abstraction through automated assistance. In summary, the improvement in efficiency permits improvements in an abstractor's bandwidth, which is significantly cost-effective. The data, which historically has been unidirectional, can now populate our own data warehouses, link registries, and we can embark on clinical research and quality improvement opportunities really enabled by comprehensive data capture. Thank you for the opportunity to share our experiences. If you have any questions or comments, please feel free to reach out to me at any time using this contact information.

automated data abstraction

natural language processing

clinical registries

electronic health records

NLP software