Medical professionals love data. The more data they have, the better care they can provide their patients. And one of the largest healthcare providers in the New York State area, Mount Sinai Health System, is taking this a step further, having formed a machine learning (ML) and predictive analytics team to help develop solutions for doctors, nurses and clinicians, to help facilitate quicker interventions during care.
Mount Sinai employees approximately 40,000 people and has eight hospitals - describing itself as one of the ‘biggest and best hospitals in the country’. I got the chance to speak with Prem Timsina, lead data science engineer of clinical science data at Mount Sinai, about how the healthcare provider is using MongoDB, as well as other open source technologies, such as Apache Spark and Scala, to build interesting ML use cases for hospital operations and care situations.
Timsina’s team was formed two years ago and was specifically hired to work on AI and ML based software for hospital operations. The first proof of concept that has been developed and is in use is for identifying patients each day that could be at risk of malnutrition. Timsina explained that this isn’t malnutrition in the traditional sense, but rather identifies patients that, for whatever reason, are finding that their bodies aren’t digesting food (rather than willingly avoiding it or eating the wrong things). He said:
In September we started to do the pilot of the malnutrition predictive engines. What it does is makes a blanket predictions of all the patients in the hospital. What is the chance the person is generating the malnutrition diagnosis? In the hospital, in patients, they might be eating food, but they are not able to digest it. So their body is not getting nutrition. If their body is not getting nutrition, then the doctors treatments or interventions may not be effective. And there may be complications.
So, every 4am we make this malnutrition prediction and file it to the EHR (Electronic Healthcare Record). When the dieticians start their shift, they look at the patient list and try to visit the persons from the highest risk score to the lowest risk score. After we deployed this application, we were able to statistically improve the number of people they are able to identify as someone that is having nutritional issues. It has a very good impact on the quality of the person’s care and the hospital’s ROI.
And by ROI, Mount Sinai is measuring the quality of care the patient receives, the time saved by hospital staff and cost savings. I asked Timsina about the accuracy of the predictive engine, but he believed this to be a misleading term. Instead, he said, his team is focused on finding “needles in the haystacks”, and so defines success parameters compared to the past or another control environment.
A new trial
Timsina’s team is this week launching its next clinical trial using the ML and AI engine, which aims to provide real-time information to doctors and nurses about whether a patient is likely to end up in an intensive care unit or an operating room in the next six hours. He explained:
This is a real-time machine learning application use case, where we have modified our early warning system. So, if you go to one of the hospitals and a nurse measures your blood pressure and inputs that into the EHR, in real-time you get that blood pressure data into our machine learning engine. We combine that information with all the past historical lab data, all the clinical data, and make a prediction - what is the chance that person will go to ICU or operating room in the next six hours?
We send back the prediction to the clinician, the nurses. If the prediction score is very high, they will do an intervention. And the idea is that if they do that intervention, it will reduce the chance that person will go to the ICU.
The results are returned in 60-70 seconds and we have the historical patient data since 2001 in MongoDB.
The clinical trial will take place over the next few months in one hospital, but it is hoped that if it proves to be successful (which Timsina believes it will, as they’ve been running tests already) then it will be rolled out to other Mount Sinai hospitals.
The reaction from hospital staff
As one can imagine, the use of predictive technologies could be met with resistance by doctors, nurses and clinicians, whom have little insight into why or how the results are being arrived at. Trust needs to be built between the machine and the healthcare provider, particularly given what is at stake.
However, Timsina and his team had a plan for this from the start. Firstly, his lead data scientist is an MD too, which helps to ensure the planned models and use cases are based in reality. Secondly, the team carries out extensive stakeholder engagement, communication and education programmes to ensure that the hospital staff are brought along on the journey. He explained:
When we start to develop these kinds of applications, what we do is we create a user group. From the inception of the programme, before we start to do any coding, before we start to develop any plan, we include the stakeholders, the actual users.
Every two weeks, or every week, we have meetings and we try to involve them as one of the members of the whole development lifecycle. During this process they know what we are developing and they also get educated about the AI and ML. It’s very iterative, small baby steps, to make them trust it.
During the process we had many discussions and many education sessions with the clinicians, nurses, and doctors. During the process everybody learnt about machine learning, how the algorithms give the probability scores, etc. Most of the time they already have the alert system in the hospital, but it’s based just off four or five parameters. Now it will be based on 300 or 400 parameters.
You’ve got to bring everybody on the same page. They are the ultimate users, so you need to help them understand what we are developing.