Data is everywhere in medicine, but it’s not living up to its promise to improve care

Published on
March 30, 2017

Recently, I approached a number of physicians and IT experts about investigating the colorectal cancer screening status of patients in our health system. Who was not screened at all? Were some patients receiving excessive testing? Why? These are relatively simple questions, but their answers could identify breakdowns in care and suggest potential solutions. My aim was to develop a novel hospital-wide collaborative to reach out to at-risk patients and find personalized ways to help them obtain the screening they needed. In doing so, I hoped to detect any cancer earlier and provide more timely and effective treatments.

I was told this data would take at least a year to acquire. Meanwhile, pilot projects were still transcribed with pen and paper. Almost four years after the installment of a new electronic medical record (EMR) system, the few software engineers on campus were overworked with requests like this. More were to be hired, but for now, the work backlog meant physicians were facing indefinite delays to obtain data for a variety of clinical and investigative projects. As one administrator told me, research is always important but keeping the hospital functioning takes priority.

Experiences like these are not unusual. There is a bizarre paradox in the culture of medicine: We generate more data than ever, but skewed priorities emphasize some data at the expense of others, and limit our ability to effectively use it. We collect ample data on doctors’ productivity, but struggle to measure and improve the quality of that care. Metrics on medical conditions such as bone marrow transplantation are rich, precise, and analyzed in real time, yet we have large gaps in our understanding of basic public health needs, such as access to preventative health. Largely through neglect, health care data has become fragmented, hard to access, and littered with inaccuracies.

This is an odd and disquieting finding for the medical field, which values precise measurement, empirical testing and the scientific method. We demand enormous amounts of information from our patients, ranging from the often-embarrassing questions we ask to frequent blood draws and imaging tests. Yet, it is still uncommon for these rich streams of data to drive research and innovations in care delivery. Doctors and nurses are overburdened by reams of often-irrelevant clinical data, but they still often lack the essential information they need to care for their patients.

There is a bizarre paradox in the culture of medicine: We generate more data than ever, but skewed priorities emphasize some data at the expense of others, and limit our ability to effectively use it. We collect ample data on doctors’ productivity, but struggle to measure and improve the quality of that care. 

Few other industries generate as many customer interactions annually as we do in health care. In 2011, there were almost 126 million outpatient visits alone in the U.S. And yet, the majority of data we gather from these visits is not put to any meaningful use. This level of ignorance would be unsustainable in almost any other business but it’s status quo in health care. There are many reasons for this but one of the biggest is that data is often used too narrowly — a way to manage patients, ensure legal compliance, and a way to obtain payment. Not surprisingly, software for billing is well developed while methods to manage clinical data, track population health outcomes and integrate biomedical research languish by comparison.

For all the hype surrounding the era of big data — massive number crunching efforts to discern novel medical findings — medicine itself is woefully under-informed when it comes to day-to-day care delivery. Big data has promise for sure, but we also need smaller data as well, and we need to better use the data we do have. In my case, this information could have revealed which patients needed colorectal screening, and made a measurable impact in the lives of real people. We need these types of insights in medicine to help to deliver the right care to the right people at the right times. This information is crucial.

Data alone without a corresponding shift in priorities will not solve the many challenges of health care delivery in the U.S. In 2015, results were published from the American College of Surgeons’ National Surgical Quality Improvement Program. Launched in 1994, it represents one of the largest attempts to gather and analyze clinical data in order to track and create benchmarks for outcomes — in this case for a number of surgeries across some 400 primarily academic hospitals. The goal of the program is to create a robust data registry to improve surgical results for a variety of procedures and minimize complications. And yet two teams of independent researchers found that enrollment in this program did not meaningfully improve outcomes on measures such as infections or mortality when compared to similar hospitals that did not participate, over a period of approximately three years.

This program has had other successes, but studies like this remind us that in medicine, innovation is comparatively easy but delivering evidence-based solutions to patients is hard. The path between measurement and outcome, care and performance is not a clear one. Medicine is not like finance, entertainment or athletics — fields that have effectively used data to boost performance and profits. Medicine is akin to education and government: messy social services that involve political agendas, large administrative and regulatory burdens, and deeply entrenched legacy systems, all serving ethnically and socioeconomically diverse populations. These complexities emphasize the pressing need to gather and use high-quality data to guide our patients’ care, and to create best practices relevant to real-world settings.

Data is more than the 1's and 0's that encode it. In health care, it represents people's lives. This is perhaps the saddest takeaway from all of this: We in medicine have allowed clinical notes and accompanying data to be largely reduced to a method to determine reimbursement, and have systematically undervalued it as a rich source of healing. When used judiciously, data can change health outcomes and improve lives, including from diseases such as colorectal cancer.

Read through patients' charts and you will see a chronicle of some of the most important moments of their lives — births, deaths, hopes for the future, and struggles with painful and often unseen maladies. Modern medical records and all the notes they contain is the largest anthology of human stories ever created. Nearly all of us will be included in it at some point in our lives. We need a better way to harness it, and to explore it for the rich meaning it contains. And that starts by valuing it as a pathway to healing before payment.


[Photo: Adam Berry/Getty Images]