Heartening life-saving applications that apply deep learning to medical data

Profile picture for user kmarko By Kurt Marko January 1, 2018
Deep learning algorithms are accelerating the pace at which medical science discovers and predicts health risks. Much more lies ahead.


Some of the most exciting, consequential and heartening applications of AI in general and deep learning, in particular, are in medicine.

As I wrote last time, many of these derive from deep learning image analysis that discovers patterns, identifies key features, then categorizes those patterns. Although social networks and consumer photo management applications have pioneered the use of deep learning for image tagging and classification, as I detailed in my last column,

Radiology has emerged as one of the most significant uses of deep learning, where the same algorithms that can tell a beagle from a boxer in your photo album can also identify tumors, tuberculosis and heart disease in medical images.

However, medical imaging isn't the only field where deep learning software promises to disrupt healthcare in a good way since its proficiency for pattern matching, trend spotting, and prediction have applications from cardiology to pharmacology. One of the most visible and potentially far-reaching opportunities pairs deep learning algorithms with the ubiquitous sensors in phones and wearables as an early-warning system for heart problems. Indeed, it's an area where two technology heavyweights, Apple and Stanford University, have joined forces on research to study and improve heart health.

Stanford heart studies: pairing mobile sensors with predictive deep learning

Even before the first generation Apple Watch was introduced, Stanford introduced an app that exploits Apple's ResearchKit platform to collect activity data from iPhone sensors that correlates to heart health. By tapping into even a fraction of the vast population of iPhone users, Stanford saw a way to rapidly accumulate a diverse data set that could be used to feed its Biomedical Data Science Initiative and provide data-driven insights into heart health. The goal was to identify previously unknown heart risk factors and provide a vehicle to promote heart-healthy activities.

The initial heart study yielded five main results:

  1. Demonstrated the feasibility of using the smartphones of consenting owners to collect medical data.
  2. Showed "that large-scale data can be gathered in real-time from mobile devices, stored securely, transferred, deidentified, and shared securely, including with participants."
  3. Collected a meaningful data set for a 6-minute walk test within weeks, far faster than when using conventional collection methods
  4. Discovered a factor mitigating heart risk, namely that individuals with a pattern of frequent changes from inactive to active states exhibited better health than inactive individuals and comparable to those with more active lifestyles.
  5. Concluded "that there is a poor association between perceived and recorded physical activity, as well as perceived and formally estimated risk."

The iPhone-based study was effectively a proof-of-concept for more meaningful opportunities using the Apple Watch with its integrated and highly accurate heart rate monitor. The first such experiment was initiated in November when the Apple-Stanford partnership released an app that uses the Watch’s heart rate sensor to collect data on irregular heart rhythms and notify users who may be experiencing atrial fibrillation (AFib). Although the announcement doesn’t specify details of the analytical technique, it’s likely an outgrowth of Stanford’s previously published research on arrhythmia detection using convolutional neural networks (CNN) with data from a single-lead EKG (a simple, portable variety that is popular for use in the home and sports training).

The Stanford team’s technique uses deep learning on 30-second EKG consisting of 6,000 sample points, with a dataset of 64,121 ECG records from 29,163 patients to produce a predictive model. Arrhythmia predictions were compared to readings from cardiologists and found to be significantly more accurate. According to its published findings, which details the CNN design and training objective function (emphasis added),

On this dataset, we train a 34-layer convolutional neural network which maps a sequence of ECG samples to a sequence of rhythm classes. Committees of board-certified cardiologists annotate a gold standard test set on which we compare the performance of our model to that of 6 other individual cardiologists. We exceed the average cardiologist performance in both recall (sensitivity) and precision (positive predictive value).

The paper mentions future work that applies the method to detect other types of arrhythmia and heart disease, including forms that are impossible to see with single-lead devices. Extending the technique to use sensor data from the Apple Watch significantly expands both the variety of data available to train predictive models and the applicability of the results by using a widely-used consumer device. Considering the tens of millions of Watch users that continuously wear the device, it means that a significant portion of the US population has access to a potentially life-saving monitor. Indeed, even Watch's basic heart rate monitoring feature, with can notify wearers when the rate spikes in a seemingly restful state, i.e. without other sensors detecting physical activity, has been credited with providing potentially life-saving early warning to heart problems, embolisms and blockages.

Many other medical applications for data analytics and AI

Stanford's Biomedical Data Science Initiative has contributed to several recent medical discoveries by mining massive troves of often unstructured data to find patterns and correlations with predictive value. For example, a recent study analyzed electronic medical records of stroke patients to assess the risk of a second catastrophic incident. The research focused on atrial fibrillation, or rapid, irregular heartbeats that can trigger a second stroke or heart failure and are commonly monitored when stroke patients are hospitalized.

Using almost 10,000 diagnosed stroke patients in the Stanford Hospital database, the researchers fed clinical data, diagnosis codes and text records from clinical notes into a model that included biomedical facts about each patient (age, weight, etc.). The model was able to find patterns and correlations among those that eventually were diagnosed with atrial fibrillation that was distilled into seven risk factors that collectively predicted which stroke patients were most likely to develop the heart condition and require aggressive monitoring.

A team from IBM’s Center from Computational Health developed a similar predictive model for heart failure using patient medical records that was at least 80% accurate out to one year in advance of an incident. The research also developed guidelines for the minimum amount and type of data needed to train similar predictive models for other diseases using longitudinal electronic health records.

Deep learning is also useful in accelerating drug discovery and precision targeting for particular diseases and individuals. The technique can be used to find patterns between the molecular structure of a specific compound and its activity in the body (so-called QSAR analysis) as well as predict how well a hypothetical molecule will bind to a receptor or other biological target (so-called docking). Given the quantity of data required, some research focuses on optimization techniques that speed the analysis. For example, Stanford researchers have developed a method that significantly reduces the amount of data needed to make meaningful predictions in drug discovery. The Deep Chem open source library aims to “democratize deep learning” via a curated set of useful algorithms, models and example code.

Several startups have emerged to exploit deep learning to accelerate drug development. For example, Insilco Medicine has partnered with GSK to identify novel and promising biological targets to accelerate drug development. Atomwise has used AI technology to develop two drugs that show promise at targeting the Ebola virus.

My take

The nexus of vast collections of electronic health records, wearable devices with sensors producing medically useful data, advances in microbiology and genomics and exponential improvements in deep learning and AI is yielding significant advances in proactive medical diagnosis and the development of precisely targeted treatments.

Given the rapid proliferation of relatively low-cost wearables, the pace of improvements in deep learning research and capabilities of accelerated hardware for deep learning algorithms, the advances so far are only a precursor to much more significant achievements. It’s an exciting time to be a medical or pharmacological researcher or data scientist.

While examples from healthcare offer some of the most compelling benefits for society, they also demonstrate the applicability of deep learning and other AI techniques to problems in other industries. As organizations continue to generate and choke on data, deep learning combined with creative thinking about a problem space and solution strategies offers a compelling way of turning unused data into business-transforming products and services.