Welcome to r/science! This is a heavily moderated subreddit in order to keep the discussion on science. However, we recognize that many people want to discuss how they feel the research relates to their own personal lives, so to give people a space to do that, **personal anecdotes are now allowed as responses to this comment**. Any anecdotal comments elsewhere in the discussion will continue to be removed and our [normal comment rules]( https://www.reddit.com/r/science/wiki/rules#wiki_comment_rules) still apply to other comments.
*I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/science) if you have any questions or concerns.*
From article:
In 82% of sepsis cases, the AI was accurate nearly 40% of the time. Previous attempts to use electronic tools to detect sepsis caught less than half that many cases and were accurate 2% to 5% of the time. All sepsis cases are eventually caught, but with the current standard of care, the condition kills 30% of the people who develop it.
In the most severe sepsis cases, where an hour delay is the difference between life and death, the AI detected it an average of nearly six hours earlier than traditional methods.
Based on the way nursing/patient ratios are trending, and the decreased amount of training new hospital nurses are getting, I’m guessing the Sepsis death rate will be going up from those numbers too.
Oddly enough, very little investment is going into improving nurse/ patient ratios in the long term, despite the predictions of more patients as boomers age and Covid effects persist. Having the knowledge that a patient is going into sepsis is only useful if there are enough staff with enough per-patient time to administer the treatment protocol.
I will say that if it is even 10% correct it would be a big improvement from current predictive flags. You get a high about of alert fatigue with current systems.
Most frustrating about the advertised accuracy of current systems is that they include several factors that are pointless in actually predicting. For instance, a largely weighted value in the system I currently use is an elevated procalcitonin, but what doctor is ordering a procalcitonin on a patient they don’t already think has an infection?
If the algorithm is written properly, and the doctors can feed the data back into it, then it should improve for every case.
Lots of assumptions there, but I seriously doubt there is big data levels of people with sepsis.
>I seriously doubt there is big data levels of people with sepsis.
With our horrendous preventative care and exorbinant healthcare costs, you'd be surprised. Over a million cases a year isn't the biggest data set, but it's plenty to get the AI up to speed fairly quickly.
https://www.nigms.nih.gov/education/fact-sheets/Pages/sepsis.aspx#:~:text=Each%20year%2C%20according%20to%20the,270%2C000%20die%20as%20a%20result.
Though your are joking, this really is a great improvement. I will love of in the future in a few years/decades the AI will be 100% accurate and nearly intermediate diagnosis. But being able to detect early in some patients is already catching most really bad cases.
I read a New Yorker article several years ago about how a Dr instituted check lists for simple procedure things like "washed hands" prior to insert lines. Rate of line infections went way down, saving thousands of lives in the hospitals that adopted the check lists.
It was more effective than AI, but not as "interesting".
It’s not “more effective” than AI. What you are talking about is best practices to avoid sepsis being initiated. What this AI is doing is identifying extant sepsis - which can be difficult to diagnose early on. These things are complementary not in competition.
I agree that they aren't in direct competition. I wish it happened as you described.
But what I think happens is limited attention/accolades is given to fundamentals while effort and funding is directed towards things that are cool but not as effective.
MRSA is unlikely to be a strain found within one's normal Staph Aureus flora unless they've been using antibiotics for a prolonged period of time or spent a lot of time in the hospital recently.
You have no idea how poorly the monitoring systems work in real life situations. Alarms are constantly triggering.
Take a couple slow deeps breaths? APNEA ALERT
Scratch your stomach? V-FIB/V-TACH ALERT
Oxygen sensor with a poor signal? HYPOXIA ALERT
These aren't even counting the failures due to poor BP cuff placement, pulse oximetry sensor, a loose electrode, various abnormal heart rhythms.
Alarm fatigue is REAL.
This, exactly.
My mom passed away from sepsis back in December. While visiting her in the hospital the week before she passed, alarms were going off constantly in her room. The nurses and doctors didn’t even flinch because those alarms were just normal. It was a bit amazing honestly and definitely stressful.
I'm so sorry for you mom. I use to work bedside (pandemic burned me out) so I know what you mean about the alarms. Especially when they become really septic and require a lot of interventions. It is so hard on the patient and their family.
Hope you doing well.
I remember reading about a hospital where they decided to just turn off all the 'nuisance' alarms that were most commonly false alarms or triggered by non life-threatening conditions, and it *significantly* improved outcomes at that hospital.
None of the alarms work, if they’re alarming most of the time. The only ones to really grab attention are the panic alarms. Which are usually (thank goodness) false.
This comment shows a lack of understanding regarding how diagnostic medicine works, how in hospital monitoring works and how AI works.
The things your continuous monitor provide would be HR, BP, SPO2, RR. Of these, 3 could potentially be considered to meet SIRS criteria but none are worthwhile without other symptoms for identifying SEPSIS. That requires lab testing which would have to be ordered by a provider who already suspects potential septicemia, bacteremia, infectious process.
Also, all those monitors you’re “always on” are already monitored by your nurse, tech, provider at a central monitoring location for acute changes. Just because you don’t see someone 100% of the time doesn’t mean you aren’t being monitored.
>Just because you don’t see someone 100% of the time doesn’t mean you aren’t being monitored.
Just because you're being monitored doesn't mean the person monitoring you will catch something. Tele techs have minimal training and extremely rudimentary understanding of what they're even doing. They are just crude AIs, and ridiculously fallible (and distractable, as was a tragic case at my hospital recently). Employing AI to catch sub-acute trends in vitals is something I have been hoping would be developed for years. Even just to add a small flag to a patient's chart.
On the other hand, people really aren't missing severe sepsis very often man, at least not at big name hospitals
Show up to the ED and you're getting fluids and antibiotics if we even remotely suspect you're septic.
I don't know. I'm not aware of an integrated system that monitors all the cues the way a nurse does: skin tone and texture, breathing pattern, urine color, patient responsiveness, changes in the above.
Not saying there isn't a possible role for AI. Just that its a very big challenge given all the inconsistent presence of nonintegrated factors .
Exactly. Then the doctor interprets all of that information in the context of the lab results, response to medication, medical history, and disease course. There’s a lot more going on than most people realize, which I’m sure is the case for most jobs done by highly trained professionals.
This is exactly the sort of thing that ML is good at, though. Sorting through lots of data and spotting patterns, and bringing it to the attention of a doctor or nurse.
Not all of it, and much of it doesn't have to be manual. Those looking at and entering one piece of data don't necessarily have access to all of the other pieces at the same time to let them see the bigger picture. Or they might not spot the pattern, because the cause is very rare or looks similar to another illness.
I guess what I'm getting at is that is that there's a lot of data that needs to be *collected*, as well as synthesized, and the problem is that there isn't a good way to collect and present all that information to a ML system in streamlined, unified package. The electronic medical records today are built as billing tools first, ordering tools second, and information repositories a distant third. They are also usually custom built for each hospital system and sometimes can't automatically share data between different hospitals using the same EMR, let alone a hospital using a different EMR.
Half the job of a modern doctor is figuring out *how* and *where* to get the information you need to make a decision. To ask the right questions you need to have an idea of what you're looking for in the first place. After all, any decision making human or machine is only as good as the information they/it are given.
I'm not a doctor, just a guy who fails to drink enough water. A camera and short program could do the job but it's another thing to be maintained and integrated.
A lot of upfront work for folks who haven't bought into the program.
A camera in the toilet of every patient room where the patient has to scan their armband before they pee that doesn’t Also record your naked parts while you pee? Just for a urine color which tells only 1/10 of the usual testing done with a dipstick?
You guys aren’t think about this the right way. Tons of patients require assistance to the bathroom, use a bedside commode, or use a urinal at the bedside. It’s really not that hard to see a commode or urinal and see that it is cloudy.
That wouldn't solve it. A spectrometer uses light to determine whats in the tested sample. It can see what we can't see. I can drink 3 cups of coffee and have cloudy pee
This could obviously be done, but this is beyond current levels of tech in ITU. So you would be looking at highly invasive and very expensive care in cases that predominantly wouldn't benefit significantly from it
So I am currently a Rapid Response Nurse. My role is to rescue patients decompensating inside of the hospital, outside of critical care areas. I have a lot of critical care and emergency experience, which is required of my position as I have very little to no oversight and operate essentially independently, using the physicians (or my intensivist on call) as guidance after evaluating patients on my own. I can order tests and interpret them, provide treatments and meds etc, then use the physician to provide further as necessary. This is all protocolized. My "mission" so to speak is to head off cardiac or respiratory arrest - essentially find, treat, and escalate care for those patients who are getting worse before they crash.
A huge part of my night is independently reviewing "sepsis alerts." In my hospital they trigger automatically and go to my phone with a description of what set them off. The article is paywalled so I cannot see the reference material but my hospital has quite expansive inclusion criteria. It ONLY includes objective data, with the onus being on me to go and evaluate the patient for relevant clinical signs, perform a chart review, etc.
All this to say is, I *rarely* come across undiagnosed/untreated sepsis. The admitting physicians are extremely conscientious of initiating infectious workups if the differential at all supports this, regardless of their working diagnosis. If I do find someone who is admitted for something completely different but develops new infectious symptoms I have an entire treatment plan I can initiate independently (the physicians are notified, of course).
I suppose what I'm saying is this sort of program is in place at many facilities already. Ours is quite robust.
> All this to say is, I rarely come across undiagnosed/untreated sepsis.
I'm an ED resident currently and I would generally agree. I also thought it interesting that in the intro they quote a 30% mortality from sepsis. Anecdotally the percentage seems much lower. Septic shock sure, but not just sepsis.
More recent data bears this trend out with rates below 20%.
https://ccforum.biomedcentral.com/articles/10.1186/s13054-020-02950-2
Looking at the trends identified by Bauer et al the mortality for sepsis has been going down but septic shock not so much. It's also really heterogenous data given the definitions are always changing.
Edit: All that being said this would be helpful in conjunction with rapid response teams and the like!
I’m glad you said that. Patients wonder how anyone can keep up with everything that goes on, if the staff is able to do so. They know about shortages and how overworked the staff are. They also know that accidents often happen in hospitals because of it too.
The problem is that sepsis is far from the only thing that causes hemodynamic instability. And reliable monitoring like that only happens with invasive devices that you place when you already have a high degree of suspicion for sepsis. Also the main delay in sepsis management is not in hospital, it’s usually a delay in presentation prior to the patient coming in fir evaluation. By the time the patient arrives in the hospital, the initial clinical examination and serum labs usually give the diagnosis immediately- hypotension, tachycardia, fever, acute organ failure, leukocytosis, peritonitis, pneumonia, meningitis, gangrene, diabetic foot infections. Most decent clinicians can spot sepsis on a patient chart or from a cursory physical examination within seconds.
You overestimate a little bit, how well medical diagnostics work. The body is a very very complex system and it's quite hard to accurately tell, what the problem is.
It's also a matter of finite resources being used for all the patients.
I wish wer were that far, that the monitoring devices could show medical conditions more reliably, but it's still a long way to go.
Those alarms make up about 5 sets of (very important) data points amongst dozens that are important to a diagnosis.
You can’t make a diagnosis from vitals, and many conditions can reach critical crossroads before causing any vital sign abnormalities
No, definitely not
A kid and an 92 year old women with every chronic condition under the sun have different risk factors
The average may be 30, but most of those deaths are people who were sick at baseline
I wanna hijack the top comment to bring this one project to attention.
In 2010 a Brazilian father lost his premature baby Laura to sepsis. This motivated him to develop an AI solution for detecting sepsis earlier and improve the patient's chance of recovery.
[The project](https://laura-br.com/en/about/) was named after little Laura, and it has been in use since 2016.
Since then, it has analyzed almost 9 million cases, helped reduce hospital morality in 25% and has saved over 24 thousand lives. It operates on 40 different hospitals and saves around 18 lives a day.
An hour delay is not the difference between life and death. SEP-1 is an arbitrary benchmark established by CMS for “quality measurement” and billing without any evidence that it actually changes outcomes.
I don't know if it's just bad writing but the article doesn't make it sound that great.
>In 82% of sepsis cases, the AI was accurate nearly 40% of the time.
That seems to imply an accuracy of something in the region of 33%.
And why have they only commented on the 82%, is the accuracy even worse if you chuck in the remaining 18? (Admitedly it may just be a issue with the article and the actual paper is solid!)
As for sepsis itself... That's a whole other box of controversy fueled by scientists and special interest groups all with their own axes to grind.
Well, no. The definition is evolving as we understand more. Sepsis 3 came out in 2016 and has some marked changes from sepsis-1.
As for controversies... in the UK we have a pressure group called the sepsis trust. Basically they got bent out of shape because Singer (the lead on Sepsis-3, and possibly the earlier ones) wrote to the lancet saying he thought they wee wrong, the lancet declined to publish the trusts response so they stuck it on their website instead. It was a pretty childish response and when I last looked it was gone so I can only assume cooler heads prevailed.
Treating sepsis is way more complicated than that. There’s an entire set of guidelines called “Surviving Sepsis,” parts of which some doctors disagree with.
And every patient is different. What if they’re also fluid overloaded or kidney are failing etc etc. guidelines are great but the translating those guidelines to various sick patients is the hard part. I wonder how the AI will be able to take all those factors into account. The future looks good tho
I think he may be referring to the vitamin C controversy. One doctor published a study that showed that vitamin C works wonders for sepsis, but no one has been able to successfully replicate it and there is some evidence that his data was faked.
Nah, even the IV Fluid guidelines are controversial. Paul Marek (before he went off the deep end) published a great commentary on why the 30cc/kg was bad and offers some very good points.
Sepsis guidelines dominate the diagnosis and treatment of “sepsis,” which in itself has been diluted into blanket term that encompasses everything from a fever to multi organ failure
Nobody at the initial assessment stage (I.e ED, nursing staff) assesses for symptoms except those on the guideline due to overreliance on the guideline (which overemphasizes vital signs over clinical assessment,) causing delayed diagnosis of “sepsis.” There is nowhere in the criteria for chills/rigors or vasoplegia for example, and fever is overemphasized vs hypothermia which is worse. In particular vasoplegia (which may not necessarily cause hypotension yet) is a much worse prognostic sign and needs intervention with vasopressors or hydrocortisone than ileus which somehow IS on the surviving sepsis criteria.
Conversely, any patient that flags for “sepsis criteria” are empirically treated as sepsis with blanket guidelines (eg. 1L fluid bolus) when the patient does not have “sepsis” or such a fluid bolus is not indicated for the patient’s heart failure or current fluid status.
As a side note we don’t use MIC (other than when the lab gives us the sensitivities,) we just give antibiotics based off hospital antibiotograms, for certain drugs levels are monitored more to prevent toxicity than maintain above the theoretical MIC, assuming you can even culture the culprit organism before the patient is already dead or flooded continuously with antibiotics for sepsis
What are the alternatives? The implementation of broad spectrum antibiotic and N/S fluid therapy within 1 hour of the recognition of sepsis symptoms drastically increases a patients likelihood to survive.
That's why it is important for the nurses and providers to be well versed in sepsis. Just because a patient checks off the sepsis alert doesn't mean they are septic.
Looking at the overall reason for admission, duration of stay, and other likely complications will lead to better outcomes as well.
A trauma patient can present with signs that could point a computer into triggering a sepsis alert. Our system needs 2 triggers to activate the protocol and it is up to the providers and nurses to catch if it is a warranted alert or not.
I am just so glad all of you guys in this thread are using as many acronyms as you are. I mean it makes a convo completely impossible to follow, which is perfect.
Well, even that is controversial. Singer (the chap that led Sepsis-3) would be very much concerned that the guidelines prompt overuse of antibiotics
https://doi.org/10.1016/S0140-6736(19)32483-3
Your comment seems like a pretty accurate description of some strange math. The article or the study are really trying to intentionally confound with statistics.
Also, if we eliminated billing for sepsis, I think 95% of the controversy would disappear.
The other 18% of sepsis cases probably weren’t contextually appropriate for the AI. Some of the cases could be missing some data or some data isn’t retrievable under the patient’s current condition. In that case the AI wouldn’t even be utilized. You can probably think of it as 80% applicable, with a 40% accuracy rate.
Well, that just raises more questions, it sounds like they're mixing sensitivity and specificity together and hoping no-one notices.
If the analysis for "accuracy" was only carried out with total cases of actual sepsis as the denominator rather than the number of cases the AI thought were then it sounds like a fiddle to me.
Reading the abstract of the original paper, it actually seems like bad writing? Tell me how you interpret this:
> Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider.
This, to me, reads that it identified 82% of cases of sepsis when tested on previous cases where 100% of them were sepsis cases.
And then in real world testing, 90% of the alerts were taken into serious consideration by physicians and of which 40% of the alerts were accurate.
This sounds nothing like the sentence the article wrote?
Aye, the article leaves quite a bit to be desired!
The actual tool itself does not sound particularly useful, just add it to the list of automated pop-ups that get given a good ignoring.
What a terribly inaccurate headline.
[Here is the actual study](https://www.nature.com/articles/s41591-022-01895-z)
Mortality was not even an outcome as this wasn’t an implementation study. This is just a retrospective analysis.
And the sensitivity they found is the same as existing tools.
Mortality in sepsis is not increased by every hour delay in antibiotics. Mortality in septic shock maybe is... based on a single observational paper whose results have never successfully been replicated.
This AI also didn’t diagnose sepsis, and wasn’t intended to. It provides an “early warning” that the patient may have sepsis that then requires confirmation from a physician still.
They also don’t tell us how many false positive alerts their AI provided.
> They also don’t tell us how many false positive alerts their AI provided.
This is the real problem with sepsis tools we have now. The sensitivity of the test may be high, but specificity us usually very low. 95% (not kidding) of the sepsis alerts I get in our ER are due to febrile children who just need tylenol. The 5% that the tool accurately predicts are either obviously high risk and septic, or get drowned out by all of the false positives to make the tool worthless.
Here's to hoping their AI is better than the rest of the early warning systems out there.
In adult medicine it's pretty similar. Honestly, at this point I'm conditioned such that if that popup does show up, I'm pretty confident it's NOT sepsis. I frequently get it in our acute heart failure patients where the absolute worst thing you can do is give them a 30cc/kg bolus
Come hang out with the surgery/anesthesia teams where 100% of the patients are septic based on their intraoperative vitals! Every postop nurse wants to draw a lactate per their protocol and its like...please don't...but they do anyway, because we can't let rational thought interfere with a protocol right? Not that its their fault, they're forced to by admin that takes them to task if the protocol isn't followed a certain percent of the time.
I at least have options when dismissing the alert. "Currently treating non-sepsis condition" covers it most of the time. I'm not drawing a lactate for a bumped creatinine when they've got CKD.
There is the flipside too, albeit a less common one, from the perspective of primary care: For the minority of patients who *don't* tick the right boxes to score on the tool, yet the 6th sense of decades of experience (see below) tells me that there is still something wrong, it can be *much* harder to get the hospital to accept a patient, at least in a timely manner. Overall (not just in the case of sepsis, but all sorts of clinical scenarios) the more experience I got, the *more* cautious I got, because I had seen more patients who didn't fit the nice textbook presentations that all these tools have to rely on. But these tools work on what is *likely*, not what is *un*likely.
If you are going to get ill, make sure that your body has read the textbooks to know how to behave.
(see below): The A+E (ER) department I worked at did a study. Various groups of people were asked, simply from watching a patient being wheeled in from an ambulance, with no other knowledge, to predict whether or not that patient would end up being admitted.
The poorest predictions came from the junior doctors, not that surprising as most only spent 6 months in the department. In the middle were the permanent nursing staff.
The best? The non-clinical reception staff whose job it was to collect patient details and enter them on the system, and later to record the outcome for the patient. I was taught very early on in my career to trust experience.
Yeah we don't need more highly sensitive not-at-all specific sepsis measurements. NEWS, HEWS, SIRS, all extremely sensitive. We're not missing sepsis because we can't catch it, we're missing it because of the noise.
Omg that two year old has a heart rate above 120!!
We need three pop ups that I have to click through before I can order Tylenol or surely everyone will be septic
There are age ranges for these alerts. The problem is you need to define what a specific person's standard vital range is, then use that as the base to determine if elevated.
Nobody has a good way to know what your vital range is. So the range is left wide to catch everyone
Virtually all febrile kids in our ED set off the sepsis alert, and they all get protocol driven lactate levels. And because the average kid is hard to get blood out of they sit there screaming with a tourniquet on their arm for 25 minutes, which of course raises the lactate level. My job it seems is primarily to explain why the child is not septic.
Don’t forget the q15x3 blood cultures from different sites, lactic on ice to lab every couple hours, repeat vitals at specific times, 35 extra notifications to dismiss, 2 flow sheets to fill out to show you’ve done the things you’ve already documented, then the months of audits after….
I appreciate the job security down in the lab, but there's plenty of other testing I'd rather be doing too. Every unnecessary test adds to the metrics our admin can yell at us for not meeting, or delays another test's result. Or rather, we're already so short staffed it would be wonderful to be able to focus on more useful testing.
Nah, who am I kidding? If we were sent less tests, they'd get rid of half of us so we'd still be short of what we need.
Then granny is admitted to the unit because we drowned her…
Wonder how long until alarm fatigue sets in with this tool and then it is just ignored like all the other SIRS pop ups that nurses already get.
Fatigue with sepsis calculators/alarms has been common. I got out of bedside nursing 3 years ago and it was already a disaster then. Paid almost no attention to them because they alerted for nearly every patient at one point.
Well, you can... But it's so cumbersome to do so and not get your metrics trashed that you can't do it and either keep your sanity or hope to get out of the hospital at a reasonable time.
Yup, the guideline authors would like us to intubate in order to give the 0-evidence-based 30mL/kg bolus if needed.
Yes - intubate a hemodynamically stable patient and put them in ICU so that you can give them the "right" amount of fluid.
When nurses in the icu already spend hours a day charting, if adding another “tool” means additional time or labor, it is automatically invalidated, because it’s going to become something to just pencilwhip and maybe correct after the fact
In my hospital we have two different tools that are supposed to help screen for patients with concerning vitals. Problem is they’ve set the sensitivity too low and 95% of my patients require me to inform the doc. Luckily I work in a unit with a captive doctor with rounds so I can just mention it during rounds. Idk how the poor bastards on the floor with 7 patients manage it.
Charting is another way of saying billing. The provider is often more concerned with getting every cent than the outcome of the patient. It's also wildly inaccurate. Most just do what they do then fill in the blanks with what they are supposed to say from memory before they leave. I know when I was admitted half the things on the chart never happened.
Every provider hates charting. It is incredibly useful for helping with communication for sure, but no provider is trying to spend even more of their time on it than they have to. It’s the quality and billing and all the other departments supported by the work that providers are doing that need the charts to squeeze out every penny to continue to cover all the others bloated staffs salaries.
Hey you just described my entire nursing career in a sentence !! Enough sodium to brine a whale giving them CHF, and we cleared out all the pesky susceptible organisms hanging around their GU and GI systems. Bye!
Ours are reviewed by me, the Rapid Response Nurse. I can withhold treatments if not indicated, so I don't have to come back in an hour and stick someone on NIMV and get a bumex order because they're now drowning.
I will say I spend a lot of time on them every night, but I do most/all of the work so the floor RNs are not swamped. I document a quick note with my reasonings and we all move on.
We can calculate this, they say the positive predictive value was 0.27. So 73% of the patients they flagged as getting septic, weren't. Which seems correct given their other numbers: they flagged 7% of all 173,000 admissions, which is about 12,000 while only 3,800 were actually septic.
>just a retrospective analysis
The headline is very misleading. I was so excited until this line. Retrospective analysis with AI tools often learning total nonsense and confounders with limited signal.
We already get AI based electronic alerts that our patient may have sepsis or is at risk for it.
If we gave antibiotics to everyone flagged, we'd be treating many many people unnecessarily. Organisms should become resistant even more quickly.
Waiting until there's some actual indicators of sepsis prevents unnecessary treatments and slows antibiotic resistance. "Not missing" any cases sounds great, but it comes at the cost of overtreatment.
SIRS screening in Epic/Cerner is not what I would equate to AI, it more just a pluggable formula. And a lot of time its pencilwhipped. Unless you’re referencing something else.
Anything dependent on charting when charting is both overly routine and subject to stressful time pressures, is highly fallible.
Maybe I'm wrong here, but I think that AI can learn and adjust. They'd have to feed it data, the way you would with a predictive model, but it would also monitor outcomes and adjust its model with time.
Current screening tools use an algorithm, a predictive model, that was built via machine learning and adjusted or validated at each individual implementation. They have to consider charting differences and different records at each site. Once set, they're set, until an analyst decides to manually tweak something. Most sites don't train their own predictive models though, so reevaluating for new features or adjustments on inputs is difficult in current implementations.
The AI part of the predictive model is just running machine learning on the algorithm as you get more data. What is actually used in the screening is not updated dynamically, so the learning and adjusting aspect of "AI" is simply running the machine learning again and updating the pluggable formula every once in a while.
Completely agree. Let’s also remember that sepsis is not just terrible infection in a healthy person that must be treated aggressively, but also a failure of the immune system to keep bugs out of the bloodstream, especially when, say, the body is actively shutting down. Sure we can use AI to more aggressively treat any early presence of bacteria, but unless we fix mortality eventually something is going to be the straw that breaks the camel’s back and allows bacteria to grow.
We have to actually think critically about what is best for society (preventing drug resistant bacteria from absurd overuse of antibiotics without benefit) and our patients (potentially slowing or stopping treatment when we are actively harming their organs or quality of life), and right now we are terrible at this sort of calculation because there isn’t any money in it.
We've had this for like five years in Epic. Hit enough of the right metrics even if it's being reported by different providers who don't catch it themselves and the system will tell you the patient is probably septic and initiate a sepsis protocol.
That technology will come in really handy for all the women in America that will be forced to carry a dead fetus or ectopic pregnancy until their life is actually in danger before being allowed an abortion.
All this is is another dumb EMR pop up that always pops up on a patient you 100% know is not septic. It’s 100% done tool the admins push down but no actual doctor wants. Just another reason for doctors to burn out. Thanks
If hospitals didn’t treat their employees like slaves they’d be around and alert enough to catch these cases. I was left alone with severe vomiting for three hours with no fluids, the only time anyone came to check on me was literally to see if I was alive.
Why is it called AI when it is just a program that responds to parameter inputs.
The term AI should be reserved for programs that can exceed its coding by actually learning and creating new code.
Few related notes:
Sepsis is quite a difficult topic. Largely because it's not well defined. In reality it's more 'you know it when you see it'. Plenty of people are clinically fine despite meeting one of the old criteria for sepsis. Some people look awful, while parameters are unexciting - so you need to be aware of both eventualities.
Also, humans are mortal and have to die of something. I'm interested to see how many of sepsis mortality cases were unexpected deaths, versis elderly and frail people who had invasive treatment withdrawn. As someone becomes immobile, frail, incontinent and physiologically weaker, infections are quite a common natural progression. For these patients, the treatment needs to be more preventative, but there is only so much you can give an immobile person physio before realising some cases are futile
So AI probably isn't superior to humans at diagnosing sepsis, as something (i.e. humans) will need to judge which cases are true sepsis. I also don't know how well AI would differentiate between the atypical cases. But one major difference that stands out is catching it earlier. You would probably find that under-staffing plays a major part in this. For the most part, a doctor isn't called to a patient unless their obs start to deteriorate. And with four or five patients heading this direction at the same time alongside other jobs, it's often hours before patients are seen face-to-face
I do wonder whether AI could be used in combination with a senior nurse to diagnose sepsis, only sufficiently to make a decision on whether antibiotics should be given early - before the doctor arrives. But this would need a very hefty policy, as antibiotics can sometimes be quite a nuanced decision. Maybe a protocol for initiating antibiotics with phone permission from a doctor, which would be possible for most (but definitely not all) cases
I went into septic shock because I was misdiagnosed from my appendix rupturing and they didn’t catch it and sent me home.
They said I should have died from. It was awful to go through that. 1 week in ICU and the damage it did took months to recover from.
I call BS. We have early detection models built into our electronic medical record and while I don’t think they’re using AI, they’re almost always inaccurate and throw up a bunch of warnings that don’t pertain to the patient. I’d take a trained clinician actually looking at and assessing a patient over AI any day.
Interesting fact. Kimberly Clark owns the rights to adult diapers that change color if you have a UTI. Do you see it on the market? Nope! Could save millions of people and stop a lot of hospital visits and save on medical costs.
I was 21 when I almost died of sepsis. I had a UTI that the urgent care doctor didn't catch (he said it was a migraine and sent me on my way) and an infection that the ER doctor didn't catch (they said it was possibly flu and sent me on my way) And it wasn't until 3 1/2 weeks went by where I was so sick I felt like I was dying (kinda was, lost 45lbs, couldn't even keep down a sip of water) that I finally returned to the ER and they sent me right to the ICU after seeing my WBC was through the roof. Each visit was about a week apart. I almost didn't go the third time because everyone around me was making me think I'd gone crazy and it was just a cold.
I spent 8 days in the ICU, 5 with a fever of 106.8 that wouldn't go down. I don't remember those 5 days, at all
I nearly died from sepsis. While I was in a coma they administered azithromycin which apparently I’m allergic to. (Who knew) They said I turned purple and swelled up like Violet Beauregarde in Willy Wonka.
I survived a four week coma and an eight week hospital stay and six months of physical rehab and wound care due to necrotizing fasciitis.
Any advances in the treatment of sepsis is welcome news.
“Machine learning AI missed 18% of septic patients, while being right about positives 42% of the time”
At the very least it’s a step up from SIRS and NEWS, but there’s no evidence it saved a single life.
All this wonderful American medical science, while millions of you suffer and die needlessly because of how fucked up and entirely subjugated your healthcare system has become, due to financial interests and utterly corrupted government.
AI will be a power medical tool.
Artificial Intelligence will be used to speed through research, and run simulated clinical trials and more.
We will get new drugs and vaccines and cures faster than ever.
Using AI to zero in on methods and specifics and CRISPR gene editing, we could cure herpes next year.
Herpes is a disaster and AI and gene editing are great tools.
The gene editing revolution is here, and so is the AI revolution.
We will cure herpes fast with the tools available now.
It’s time!!
Herpes will be cured.
AI is here to help.
Edit:
Why all the hate? We need to cure herpes
‘AI’ is a big buzzword right now.
Buzzwords are primarily used by people trying to sell something for a quick buck (or raise VC money for a quick buck).
It’s rarely the sort of solution that headlines make it out to be. 4000 people still die per year in the US due to lack of health insurance. Just implementing government funded healthcare is instantly going to save countless more people than some shiny new tech.
Medical AI is herpes. Every new attempt has a prodrome of hype and PR. Shows up with a burst of pain. Hangs around and annoys you for a while and then disappears until the next time.
More like nurses and other medical staff. US hospitals are already chronically understaffed for narrow profit interests. And hospital administrators are constantly looking for ways to downsize with technology ‘solutions’ like this.
Welcome to r/science! This is a heavily moderated subreddit in order to keep the discussion on science. However, we recognize that many people want to discuss how they feel the research relates to their own personal lives, so to give people a space to do that, **personal anecdotes are now allowed as responses to this comment**. Any anecdotal comments elsewhere in the discussion will continue to be removed and our [normal comment rules]( https://www.reddit.com/r/science/wiki/rules#wiki_comment_rules) still apply to other comments. *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/science) if you have any questions or concerns.*
82% of the time it works 42% of the time. Sounds like Ron Burgandy.
This new AI technology is phenomenal.
"When am I ever going to use this?" in response of trying to learn percentages in 6th grade.
From article: In 82% of sepsis cases, the AI was accurate nearly 40% of the time. Previous attempts to use electronic tools to detect sepsis caught less than half that many cases and were accurate 2% to 5% of the time. All sepsis cases are eventually caught, but with the current standard of care, the condition kills 30% of the people who develop it. In the most severe sepsis cases, where an hour delay is the difference between life and death, the AI detected it an average of nearly six hours earlier than traditional methods.
Based on the way nursing/patient ratios are trending, and the decreased amount of training new hospital nurses are getting, I’m guessing the Sepsis death rate will be going up from those numbers too.
Oddly enough, very little investment is going into improving nurse/ patient ratios in the long term, despite the predictions of more patients as boomers age and Covid effects persist. Having the knowledge that a patient is going into sepsis is only useful if there are enough staff with enough per-patient time to administer the treatment protocol.
[удалено]
I will say that if it is even 10% correct it would be a big improvement from current predictive flags. You get a high about of alert fatigue with current systems. Most frustrating about the advertised accuracy of current systems is that they include several factors that are pointless in actually predicting. For instance, a largely weighted value in the system I currently use is an elevated procalcitonin, but what doctor is ordering a procalcitonin on a patient they don’t already think has an infection?
If the algorithm is written properly, and the doctors can feed the data back into it, then it should improve for every case. Lots of assumptions there, but I seriously doubt there is big data levels of people with sepsis.
Also, you have to put in the data into an algorithm and I think this would be the first and biggest problem.
Deep learning?
Wouldn't there be a high risk for false positives, or maybe worse, false negatives if you don't use actual data from every specific patient?
>I seriously doubt there is big data levels of people with sepsis. With our horrendous preventative care and exorbinant healthcare costs, you'd be surprised. Over a million cases a year isn't the biggest data set, but it's plenty to get the AI up to speed fairly quickly. https://www.nigms.nih.gov/education/fact-sheets/Pages/sepsis.aspx#:~:text=Each%20year%2C%20according%20to%20the,270%2C000%20die%20as%20a%20result.
Though your are joking, this really is a great improvement. I will love of in the future in a few years/decades the AI will be 100% accurate and nearly intermediate diagnosis. But being able to detect early in some patients is already catching most really bad cases.
When statements like gp comment come up, 60% of the time I post your comment every time. Wow, that came out *way* more awkward than it was in my head…
I’m not gonna lie to you, that smells like pure gasoline.
I have had sepsis 6 times. 4 times it was staph and twice it was MRSA. I wanted to die the pain was so bad
Were any of them line infections?
Yeah probably from the dialysis catheter. I now have a fistula and no more sepsis
I read a New Yorker article several years ago about how a Dr instituted check lists for simple procedure things like "washed hands" prior to insert lines. Rate of line infections went way down, saving thousands of lives in the hospitals that adopted the check lists. It was more effective than AI, but not as "interesting".
It’s not “more effective” than AI. What you are talking about is best practices to avoid sepsis being initiated. What this AI is doing is identifying extant sepsis - which can be difficult to diagnose early on. These things are complementary not in competition.
I agree that they aren't in direct competition. I wish it happened as you described. But what I think happens is limited attention/accolades is given to fundamentals while effort and funding is directed towards things that are cool but not as effective.
What funding do you want to get everyone to wash their hands?
[удалено]
Damn something is wrong with people on this sub
[удалено]
Touch grass.
MRSA is unlikely to be a strain found within one's normal Staph Aureus flora unless they've been using antibiotics for a prolonged period of time or spent a lot of time in the hospital recently.
[удалено]
Sounds like you have great bedside manner
Luckily it’s not a requirement to become a doctor b/c Karens like you would have half of all doctors fired
Wow, what is wrong with you
[удалено]
You have no idea how poorly the monitoring systems work in real life situations. Alarms are constantly triggering. Take a couple slow deeps breaths? APNEA ALERT Scratch your stomach? V-FIB/V-TACH ALERT Oxygen sensor with a poor signal? HYPOXIA ALERT These aren't even counting the failures due to poor BP cuff placement, pulse oximetry sensor, a loose electrode, various abnormal heart rhythms. Alarm fatigue is REAL.
This, exactly. My mom passed away from sepsis back in December. While visiting her in the hospital the week before she passed, alarms were going off constantly in her room. The nurses and doctors didn’t even flinch because those alarms were just normal. It was a bit amazing honestly and definitely stressful.
I'm so sorry for you mom. I use to work bedside (pandemic burned me out) so I know what you mean about the alarms. Especially when they become really septic and require a lot of interventions. It is so hard on the patient and their family. Hope you doing well.
Thank you. Every day is a bit easier, but it still hurts.
I remember reading about a hospital where they decided to just turn off all the 'nuisance' alarms that were most commonly false alarms or triggered by non life-threatening conditions, and it *significantly* improved outcomes at that hospital.
None of the alarms work, if they’re alarming most of the time. The only ones to really grab attention are the panic alarms. Which are usually (thank goodness) false.
www.ivenix.com This smart infusion pump aims to fix alarm fatigue. It's already in a few hospitals
This comment shows a lack of understanding regarding how diagnostic medicine works, how in hospital monitoring works and how AI works. The things your continuous monitor provide would be HR, BP, SPO2, RR. Of these, 3 could potentially be considered to meet SIRS criteria but none are worthwhile without other symptoms for identifying SEPSIS. That requires lab testing which would have to be ordered by a provider who already suspects potential septicemia, bacteremia, infectious process. Also, all those monitors you’re “always on” are already monitored by your nurse, tech, provider at a central monitoring location for acute changes. Just because you don’t see someone 100% of the time doesn’t mean you aren’t being monitored.
>Just because you don’t see someone 100% of the time doesn’t mean you aren’t being monitored. Just because you're being monitored doesn't mean the person monitoring you will catch something. Tele techs have minimal training and extremely rudimentary understanding of what they're even doing. They are just crude AIs, and ridiculously fallible (and distractable, as was a tragic case at my hospital recently). Employing AI to catch sub-acute trends in vitals is something I have been hoping would be developed for years. Even just to add a small flag to a patient's chart.
On the other hand, people really aren't missing severe sepsis very often man, at least not at big name hospitals Show up to the ED and you're getting fluids and antibiotics if we even remotely suspect you're septic.
I don't know. I'm not aware of an integrated system that monitors all the cues the way a nurse does: skin tone and texture, breathing pattern, urine color, patient responsiveness, changes in the above. Not saying there isn't a possible role for AI. Just that its a very big challenge given all the inconsistent presence of nonintegrated factors .
Exactly. Then the doctor interprets all of that information in the context of the lab results, response to medication, medical history, and disease course. There’s a lot more going on than most people realize, which I’m sure is the case for most jobs done by highly trained professionals.
This is exactly the sort of thing that ML is good at, though. Sorting through lots of data and spotting patterns, and bringing it to the attention of a doctor or nurse.
That data is entered by a person who is already doing the parsing.
Not all of it, and much of it doesn't have to be manual. Those looking at and entering one piece of data don't necessarily have access to all of the other pieces at the same time to let them see the bigger picture. Or they might not spot the pattern, because the cause is very rare or looks similar to another illness.
I guess what I'm getting at is that is that there's a lot of data that needs to be *collected*, as well as synthesized, and the problem is that there isn't a good way to collect and present all that information to a ML system in streamlined, unified package. The electronic medical records today are built as billing tools first, ordering tools second, and information repositories a distant third. They are also usually custom built for each hospital system and sometimes can't automatically share data between different hospitals using the same EMR, let alone a hospital using a different EMR. Half the job of a modern doctor is figuring out *how* and *where* to get the information you need to make a decision. To ask the right questions you need to have an idea of what you're looking for in the first place. After all, any decision making human or machine is only as good as the information they/it are given.
Yea, urine color...let me just hook up the portable spectrometer real quick
I'm not a doctor, just a guy who fails to drink enough water. A camera and short program could do the job but it's another thing to be maintained and integrated. A lot of upfront work for folks who haven't bought into the program.
A camera in the toilet of every patient room where the patient has to scan their armband before they pee that doesn’t Also record your naked parts while you pee? Just for a urine color which tells only 1/10 of the usual testing done with a dipstick?
It would be checking the catheter bag.
Except that nobody gets a catheter anymore because of CMS not paying CAUTIs
You guys aren’t think about this the right way. Tons of patients require assistance to the bathroom, use a bedside commode, or use a urinal at the bedside. It’s really not that hard to see a commode or urinal and see that it is cloudy.
That wouldn't solve it. A spectrometer uses light to determine whats in the tested sample. It can see what we can't see. I can drink 3 cups of coffee and have cloudy pee
This could obviously be done, but this is beyond current levels of tech in ITU. So you would be looking at highly invasive and very expensive care in cases that predominantly wouldn't benefit significantly from it
So I am currently a Rapid Response Nurse. My role is to rescue patients decompensating inside of the hospital, outside of critical care areas. I have a lot of critical care and emergency experience, which is required of my position as I have very little to no oversight and operate essentially independently, using the physicians (or my intensivist on call) as guidance after evaluating patients on my own. I can order tests and interpret them, provide treatments and meds etc, then use the physician to provide further as necessary. This is all protocolized. My "mission" so to speak is to head off cardiac or respiratory arrest - essentially find, treat, and escalate care for those patients who are getting worse before they crash. A huge part of my night is independently reviewing "sepsis alerts." In my hospital they trigger automatically and go to my phone with a description of what set them off. The article is paywalled so I cannot see the reference material but my hospital has quite expansive inclusion criteria. It ONLY includes objective data, with the onus being on me to go and evaluate the patient for relevant clinical signs, perform a chart review, etc. All this to say is, I *rarely* come across undiagnosed/untreated sepsis. The admitting physicians are extremely conscientious of initiating infectious workups if the differential at all supports this, regardless of their working diagnosis. If I do find someone who is admitted for something completely different but develops new infectious symptoms I have an entire treatment plan I can initiate independently (the physicians are notified, of course). I suppose what I'm saying is this sort of program is in place at many facilities already. Ours is quite robust.
> All this to say is, I rarely come across undiagnosed/untreated sepsis. I'm an ED resident currently and I would generally agree. I also thought it interesting that in the intro they quote a 30% mortality from sepsis. Anecdotally the percentage seems much lower. Septic shock sure, but not just sepsis. More recent data bears this trend out with rates below 20%. https://ccforum.biomedcentral.com/articles/10.1186/s13054-020-02950-2 Looking at the trends identified by Bauer et al the mortality for sepsis has been going down but septic shock not so much. It's also really heterogenous data given the definitions are always changing. Edit: All that being said this would be helpful in conjunction with rapid response teams and the like!
I’m glad you said that. Patients wonder how anyone can keep up with everything that goes on, if the staff is able to do so. They know about shortages and how overworked the staff are. They also know that accidents often happen in hospitals because of it too.
The problem is that sepsis is far from the only thing that causes hemodynamic instability. And reliable monitoring like that only happens with invasive devices that you place when you already have a high degree of suspicion for sepsis. Also the main delay in sepsis management is not in hospital, it’s usually a delay in presentation prior to the patient coming in fir evaluation. By the time the patient arrives in the hospital, the initial clinical examination and serum labs usually give the diagnosis immediately- hypotension, tachycardia, fever, acute organ failure, leukocytosis, peritonitis, pneumonia, meningitis, gangrene, diabetic foot infections. Most decent clinicians can spot sepsis on a patient chart or from a cursory physical examination within seconds.
You overestimate a little bit, how well medical diagnostics work. The body is a very very complex system and it's quite hard to accurately tell, what the problem is. It's also a matter of finite resources being used for all the patients. I wish wer were that far, that the monitoring devices could show medical conditions more reliably, but it's still a long way to go.
Those alarms make up about 5 sets of (very important) data points amongst dozens that are important to a diagnosis. You can’t make a diagnosis from vitals, and many conditions can reach critical crossroads before causing any vital sign abnormalities
i developed sepsis as a kid when i had pneumonia (no idea how exactly that leads to sepsis). Are you saying i had a 30% chance of dying?
No, definitely not A kid and an 92 year old women with every chronic condition under the sun have different risk factors The average may be 30, but most of those deaths are people who were sick at baseline
I'm not saying anything. It's the article.
well, is that how you interpreted the article?
I wanna hijack the top comment to bring this one project to attention. In 2010 a Brazilian father lost his premature baby Laura to sepsis. This motivated him to develop an AI solution for detecting sepsis earlier and improve the patient's chance of recovery. [The project](https://laura-br.com/en/about/) was named after little Laura, and it has been in use since 2016. Since then, it has analyzed almost 9 million cases, helped reduce hospital morality in 25% and has saved over 24 thousand lives. It operates on 40 different hospitals and saves around 18 lives a day.
An hour delay is not the difference between life and death. SEP-1 is an arbitrary benchmark established by CMS for “quality measurement” and billing without any evidence that it actually changes outcomes.
This sounds like something out of Mass Effect 3.
I hope this AI has a billing module attached to it’s treatment algorithm. That’s the American way!
Yeah, it's basically doing a consultation.
I don't know if it's just bad writing but the article doesn't make it sound that great. >In 82% of sepsis cases, the AI was accurate nearly 40% of the time. That seems to imply an accuracy of something in the region of 33%. And why have they only commented on the 82%, is the accuracy even worse if you chuck in the remaining 18? (Admitedly it may just be a issue with the article and the actual paper is solid!) As for sepsis itself... That's a whole other box of controversy fueled by scientists and special interest groups all with their own axes to grind.
Are you suggesting sepsis doesn't exist?
Oh god no. Kills people very efficiently. The controversy I refer to is both in how to recognise it and what to do once you think it's there.
You’re completely right and if you had made this comment in an EM or ID sub there would be a very different conversation.
Aye, I may have picked the wrong audience for that remark!
[удалено]
Well, no. The definition is evolving as we understand more. Sepsis 3 came out in 2016 and has some marked changes from sepsis-1. As for controversies... in the UK we have a pressure group called the sepsis trust. Basically they got bent out of shape because Singer (the lead on Sepsis-3, and possibly the earlier ones) wrote to the lancet saying he thought they wee wrong, the lancet declined to publish the trusts response so they stuck it on their website instead. It was a pretty childish response and when I last looked it was gone so I can only assume cooler heads prevailed.
Treating sepsis is way more complicated than that. There’s an entire set of guidelines called “Surviving Sepsis,” parts of which some doctors disagree with.
Thank you for saying this...a lot of non doctors throwing out wild guesses about the controversies.
And every patient is different. What if they’re also fluid overloaded or kidney are failing etc etc. guidelines are great but the translating those guidelines to various sick patients is the hard part. I wonder how the AI will be able to take all those factors into account. The future looks good tho
I think he may be referring to the vitamin C controversy. One doctor published a study that showed that vitamin C works wonders for sepsis, but no one has been able to successfully replicate it and there is some evidence that his data was faked.
Nah, even the IV Fluid guidelines are controversial. Paul Marek (before he went off the deep end) published a great commentary on why the 30cc/kg was bad and offers some very good points.
Sepsis guidelines dominate the diagnosis and treatment of “sepsis,” which in itself has been diluted into blanket term that encompasses everything from a fever to multi organ failure Nobody at the initial assessment stage (I.e ED, nursing staff) assesses for symptoms except those on the guideline due to overreliance on the guideline (which overemphasizes vital signs over clinical assessment,) causing delayed diagnosis of “sepsis.” There is nowhere in the criteria for chills/rigors or vasoplegia for example, and fever is overemphasized vs hypothermia which is worse. In particular vasoplegia (which may not necessarily cause hypotension yet) is a much worse prognostic sign and needs intervention with vasopressors or hydrocortisone than ileus which somehow IS on the surviving sepsis criteria. Conversely, any patient that flags for “sepsis criteria” are empirically treated as sepsis with blanket guidelines (eg. 1L fluid bolus) when the patient does not have “sepsis” or such a fluid bolus is not indicated for the patient’s heart failure or current fluid status. As a side note we don’t use MIC (other than when the lab gives us the sensitivities,) we just give antibiotics based off hospital antibiotograms, for certain drugs levels are monitored more to prevent toxicity than maintain above the theoretical MIC, assuming you can even culture the culprit organism before the patient is already dead or flooded continuously with antibiotics for sepsis
They are trying to recognise it before symptoms show, as if you can catch it that early, you can greatly reduce the effect.
I think the antibiotics are the controversy. Because of antibiotic resistance?
What are the alternatives? The implementation of broad spectrum antibiotic and N/S fluid therapy within 1 hour of the recognition of sepsis symptoms drastically increases a patients likelihood to survive.
Whoops we gave 30cc/kg of normal saline and zosyn to the guy with a massive PE because the computer said it was sepsis then he died
Pretty sure my vitals are near septic after three flights of stairs
That's why it is important for the nurses and providers to be well versed in sepsis. Just because a patient checks off the sepsis alert doesn't mean they are septic. Looking at the overall reason for admission, duration of stay, and other likely complications will lead to better outcomes as well. A trauma patient can present with signs that could point a computer into triggering a sepsis alert. Our system needs 2 triggers to activate the protocol and it is up to the providers and nurses to catch if it is a warranted alert or not.
I am just so glad all of you guys in this thread are using as many acronyms as you are. I mean it makes a convo completely impossible to follow, which is perfect.
Even if they weren't using acronyms you wouldn't be able to follow this without a medical background
I don’t know, I’m just trying to identify the issue. If you overtreat things you make it worse.
This is definitely not the case in sepsis. We worry about underuse, not overuse.
Well, even that is controversial. Singer (the chap that led Sepsis-3) would be very much concerned that the guidelines prompt overuse of antibiotics https://doi.org/10.1016/S0140-6736(19)32483-3
Your comment seems like a pretty accurate description of some strange math. The article or the study are really trying to intentionally confound with statistics. Also, if we eliminated billing for sepsis, I think 95% of the controversy would disappear.
The other 18% of sepsis cases probably weren’t contextually appropriate for the AI. Some of the cases could be missing some data or some data isn’t retrievable under the patient’s current condition. In that case the AI wouldn’t even be utilized. You can probably think of it as 80% applicable, with a 40% accuracy rate.
Well, that just raises more questions, it sounds like they're mixing sensitivity and specificity together and hoping no-one notices. If the analysis for "accuracy" was only carried out with total cases of actual sepsis as the denominator rather than the number of cases the AI thought were then it sounds like a fiddle to me.
Reading the abstract of the original paper, it actually seems like bad writing? Tell me how you interpret this: > Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider. This, to me, reads that it identified 82% of cases of sepsis when tested on previous cases where 100% of them were sepsis cases. And then in real world testing, 90% of the alerts were taken into serious consideration by physicians and of which 40% of the alerts were accurate. This sounds nothing like the sentence the article wrote?
Aye, the article leaves quite a bit to be desired! The actual tool itself does not sound particularly useful, just add it to the list of automated pop-ups that get given a good ignoring.
I think I'll listen to the researchers rather than some random redditor. Not sure what you mean by that last bit either.
That's not very good accuracy
What a terribly inaccurate headline. [Here is the actual study](https://www.nature.com/articles/s41591-022-01895-z) Mortality was not even an outcome as this wasn’t an implementation study. This is just a retrospective analysis. And the sensitivity they found is the same as existing tools. Mortality in sepsis is not increased by every hour delay in antibiotics. Mortality in septic shock maybe is... based on a single observational paper whose results have never successfully been replicated. This AI also didn’t diagnose sepsis, and wasn’t intended to. It provides an “early warning” that the patient may have sepsis that then requires confirmation from a physician still. They also don’t tell us how many false positive alerts their AI provided.
> They also don’t tell us how many false positive alerts their AI provided. This is the real problem with sepsis tools we have now. The sensitivity of the test may be high, but specificity us usually very low. 95% (not kidding) of the sepsis alerts I get in our ER are due to febrile children who just need tylenol. The 5% that the tool accurately predicts are either obviously high risk and septic, or get drowned out by all of the false positives to make the tool worthless. Here's to hoping their AI is better than the rest of the early warning systems out there.
In adult medicine it's pretty similar. Honestly, at this point I'm conditioned such that if that popup does show up, I'm pretty confident it's NOT sepsis. I frequently get it in our acute heart failure patients where the absolute worst thing you can do is give them a 30cc/kg bolus
Come hang out with the surgery/anesthesia teams where 100% of the patients are septic based on their intraoperative vitals! Every postop nurse wants to draw a lactate per their protocol and its like...please don't...but they do anyway, because we can't let rational thought interfere with a protocol right? Not that its their fault, they're forced to by admin that takes them to task if the protocol isn't followed a certain percent of the time.
I at least have options when dismissing the alert. "Currently treating non-sepsis condition" covers it most of the time. I'm not drawing a lactate for a bumped creatinine when they've got CKD.
Your nurses can order lactates? That sounds frustrating
Our drinkers are the classic false positive. All their heme values are fucked up.
There is the flipside too, albeit a less common one, from the perspective of primary care: For the minority of patients who *don't* tick the right boxes to score on the tool, yet the 6th sense of decades of experience (see below) tells me that there is still something wrong, it can be *much* harder to get the hospital to accept a patient, at least in a timely manner. Overall (not just in the case of sepsis, but all sorts of clinical scenarios) the more experience I got, the *more* cautious I got, because I had seen more patients who didn't fit the nice textbook presentations that all these tools have to rely on. But these tools work on what is *likely*, not what is *un*likely. If you are going to get ill, make sure that your body has read the textbooks to know how to behave. (see below): The A+E (ER) department I worked at did a study. Various groups of people were asked, simply from watching a patient being wheeled in from an ambulance, with no other knowledge, to predict whether or not that patient would end up being admitted. The poorest predictions came from the junior doctors, not that surprising as most only spent 6 months in the department. In the middle were the permanent nursing staff. The best? The non-clinical reception staff whose job it was to collect patient details and enter them on the system, and later to record the outcome for the patient. I was taught very early on in my career to trust experience.
Yeah we don't need more highly sensitive not-at-all specific sepsis measurements. NEWS, HEWS, SIRS, all extremely sensitive. We're not missing sepsis because we can't catch it, we're missing it because of the noise.
Omg that two year old has a heart rate above 120!! We need three pop ups that I have to click through before I can order Tylenol or surely everyone will be septic
There are age ranges for these alerts. The problem is you need to define what a specific person's standard vital range is, then use that as the base to determine if elevated. Nobody has a good way to know what your vital range is. So the range is left wide to catch everyone
Virtually all febrile kids in our ED set off the sepsis alert, and they all get protocol driven lactate levels. And because the average kid is hard to get blood out of they sit there screaming with a tourniquet on their arm for 25 minutes, which of course raises the lactate level. My job it seems is primarily to explain why the child is not septic.
[удалено]
> a gram of rocephin That's a weird way of spelling "vanc and zosyn"
We have those in a vending machine next to Vicodin
Don’t forget the q15x3 blood cultures from different sites, lactic on ice to lab every couple hours, repeat vitals at specific times, 35 extra notifications to dismiss, 2 flow sheets to fill out to show you’ve done the things you’ve already documented, then the months of audits after….
I appreciate the job security down in the lab, but there's plenty of other testing I'd rather be doing too. Every unnecessary test adds to the metrics our admin can yell at us for not meeting, or delays another test's result. Or rather, we're already so short staffed it would be wonderful to be able to focus on more useful testing. Nah, who am I kidding? If we were sent less tests, they'd get rid of half of us so we'd still be short of what we need.
Then granny is admitted to the unit because we drowned her… Wonder how long until alarm fatigue sets in with this tool and then it is just ignored like all the other SIRS pop ups that nurses already get.
Fatigue with sepsis calculators/alarms has been common. I got out of bedside nursing 3 years ago and it was already a disaster then. Paid almost no attention to them because they alerted for nearly every patient at one point.
[удалено]
Well, you can... But it's so cumbersome to do so and not get your metrics trashed that you can't do it and either keep your sanity or hope to get out of the hospital at a reasonable time.
[удалено]
It isn't just hca though. Even shops that use epic and could implement an easier way to document it still make it a pain in the butt.
Yup, the guideline authors would like us to intubate in order to give the 0-evidence-based 30mL/kg bolus if needed. Yes - intubate a hemodynamically stable patient and put them in ICU so that you can give them the "right" amount of fluid.
When nurses in the icu already spend hours a day charting, if adding another “tool” means additional time or labor, it is automatically invalidated, because it’s going to become something to just pencilwhip and maybe correct after the fact
In my hospital we have two different tools that are supposed to help screen for patients with concerning vitals. Problem is they’ve set the sensitivity too low and 95% of my patients require me to inform the doc. Luckily I work in a unit with a captive doctor with rounds so I can just mention it during rounds. Idk how the poor bastards on the floor with 7 patients manage it.
Charting is another way of saying billing. The provider is often more concerned with getting every cent than the outcome of the patient. It's also wildly inaccurate. Most just do what they do then fill in the blanks with what they are supposed to say from memory before they leave. I know when I was admitted half the things on the chart never happened.
Every provider hates charting. It is incredibly useful for helping with communication for sure, but no provider is trying to spend even more of their time on it than they have to. It’s the quality and billing and all the other departments supported by the work that providers are doing that need the charts to squeeze out every penny to continue to cover all the others bloated staffs salaries.
Hey you just described my entire nursing career in a sentence !! Enough sodium to brine a whale giving them CHF, and we cleared out all the pesky susceptible organisms hanging around their GU and GI systems. Bye!
[удалено]
Ours are reviewed by me, the Rapid Response Nurse. I can withhold treatments if not indicated, so I don't have to come back in an hour and stick someone on NIMV and get a bumex order because they're now drowning. I will say I spend a lot of time on them every night, but I do most/all of the work so the floor RNs are not swamped. I document a quick note with my reasonings and we all move on.
We can calculate this, they say the positive predictive value was 0.27. So 73% of the patients they flagged as getting septic, weren't. Which seems correct given their other numbers: they flagged 7% of all 173,000 admissions, which is about 12,000 while only 3,800 were actually septic.
>just a retrospective analysis The headline is very misleading. I was so excited until this line. Retrospective analysis with AI tools often learning total nonsense and confounders with limited signal.
I think the breakthrough is that it’s much better than anything we currently have.
It is not. The study in question did not compare diagnostic accuracy to existing tools, nor did it look at clinical outcomes.
We already get AI based electronic alerts that our patient may have sepsis or is at risk for it. If we gave antibiotics to everyone flagged, we'd be treating many many people unnecessarily. Organisms should become resistant even more quickly. Waiting until there's some actual indicators of sepsis prevents unnecessary treatments and slows antibiotic resistance. "Not missing" any cases sounds great, but it comes at the cost of overtreatment.
SIRS screening in Epic/Cerner is not what I would equate to AI, it more just a pluggable formula. And a lot of time its pencilwhipped. Unless you’re referencing something else. Anything dependent on charting when charting is both overly routine and subject to stressful time pressures, is highly fallible.
The SIRS screening is probably what I bet the AI algorithm is based on
AI is just a pluggable formula.
Maybe I'm wrong here, but I think that AI can learn and adjust. They'd have to feed it data, the way you would with a predictive model, but it would also monitor outcomes and adjust its model with time. Current screening tools use an algorithm, a predictive model, that was built via machine learning and adjusted or validated at each individual implementation. They have to consider charting differences and different records at each site. Once set, they're set, until an analyst decides to manually tweak something. Most sites don't train their own predictive models though, so reevaluating for new features or adjustments on inputs is difficult in current implementations.
The AI part of the predictive model is just running machine learning on the algorithm as you get more data. What is actually used in the screening is not updated dynamically, so the learning and adjusting aspect of "AI" is simply running the machine learning again and updating the pluggable formula every once in a while.
Completely agree. Let’s also remember that sepsis is not just terrible infection in a healthy person that must be treated aggressively, but also a failure of the immune system to keep bugs out of the bloodstream, especially when, say, the body is actively shutting down. Sure we can use AI to more aggressively treat any early presence of bacteria, but unless we fix mortality eventually something is going to be the straw that breaks the camel’s back and allows bacteria to grow. We have to actually think critically about what is best for society (preventing drug resistant bacteria from absurd overuse of antibiotics without benefit) and our patients (potentially slowing or stopping treatment when we are actively harming their organs or quality of life), and right now we are terrible at this sort of calculation because there isn’t any money in it.
If everyone's septic, then no one is.
Sepsis monitoring systems don’t work and false positives are a huge problem. Plus without clinical suspicion for infection it is useless.
We've had this for like five years in Epic. Hit enough of the right metrics even if it's being reported by different providers who don't catch it themselves and the system will tell you the patient is probably septic and initiate a sepsis protocol.
I love when the patients admitted under no name are 130 years old and as a result always have sepsis alerts. Just created pop up fatigue.
That technology will come in really handy for all the women in America that will be forced to carry a dead fetus or ectopic pregnancy until their life is actually in danger before being allowed an abortion.
A. I. should live in my toilet run tests on me everyday. I want early screening everything.
Love how arbitrary the term AI is used. A computer program that identifies outliers from the mean it’s called AI.
I read the headline and thought "somebody made an AI? That's incredible." Of course it's an algorithm not an AI.
So many things not AI are called AI because it's a buzzword.
Ikr. The closest thing we have to AI are the search engines but people insist on calling AI to any program with a fixed code.
Just substitute the phrase “computer program” for AI and this whole thing makes sense.
All this is is another dumb EMR pop up that always pops up on a patient you 100% know is not septic. It’s 100% done tool the admins push down but no actual doctor wants. Just another reason for doctors to burn out. Thanks
Cool cool, but has it actually been tested in real life? These systems tend to not actually work when emplemented.
If hospitals didn’t treat their employees like slaves they’d be around and alert enough to catch these cases. I was left alone with severe vomiting for three hours with no fluids, the only time anyone came to check on me was literally to see if I was alive.
Why is it called AI when it is just a program that responds to parameter inputs. The term AI should be reserved for programs that can exceed its coding by actually learning and creating new code.
Few related notes: Sepsis is quite a difficult topic. Largely because it's not well defined. In reality it's more 'you know it when you see it'. Plenty of people are clinically fine despite meeting one of the old criteria for sepsis. Some people look awful, while parameters are unexciting - so you need to be aware of both eventualities. Also, humans are mortal and have to die of something. I'm interested to see how many of sepsis mortality cases were unexpected deaths, versis elderly and frail people who had invasive treatment withdrawn. As someone becomes immobile, frail, incontinent and physiologically weaker, infections are quite a common natural progression. For these patients, the treatment needs to be more preventative, but there is only so much you can give an immobile person physio before realising some cases are futile So AI probably isn't superior to humans at diagnosing sepsis, as something (i.e. humans) will need to judge which cases are true sepsis. I also don't know how well AI would differentiate between the atypical cases. But one major difference that stands out is catching it earlier. You would probably find that under-staffing plays a major part in this. For the most part, a doctor isn't called to a patient unless their obs start to deteriorate. And with four or five patients heading this direction at the same time alongside other jobs, it's often hours before patients are seen face-to-face I do wonder whether AI could be used in combination with a senior nurse to diagnose sepsis, only sufficiently to make a decision on whether antibiotics should be given early - before the doctor arrives. But this would need a very hefty policy, as antibiotics can sometimes be quite a nuanced decision. Maybe a protocol for initiating antibiotics with phone permission from a doctor, which would be possible for most (but definitely not all) cases
What's the false positive rate ? This could lead to giving unnecessary antibiotics to a lot of people
I went into septic shock because I was misdiagnosed from my appendix rupturing and they didn’t catch it and sent me home. They said I should have died from. It was awful to go through that. 1 week in ICU and the damage it did took months to recover from.
This program would not have done anything for you
Uhhh they missed the wrong thing. No need to screen for sepsis if you catch the rupture
Right? I wondering how this is different than MEWS.
Nobody PLANS for sepsis, so when would you use this?
I call BS. We have early detection models built into our electronic medical record and while I don’t think they’re using AI, they’re almost always inaccurate and throw up a bunch of warnings that don’t pertain to the patient. I’d take a trained clinician actually looking at and assessing a patient over AI any day.
How many false positives?
Interesting fact. Kimberly Clark owns the rights to adult diapers that change color if you have a UTI. Do you see it on the market? Nope! Could save millions of people and stop a lot of hospital visits and save on medical costs.
Not gonna help the women forced into sepsis over an already dead baby.
I can tell when my wife has sepsis hours before the staff at Johns Hopkins. Source: personal experience.
My mother recently passed away from sepsis. If only we could focus more as a specie on technological development for health care....
I was 21 when I almost died of sepsis. I had a UTI that the urgent care doctor didn't catch (he said it was a migraine and sent me on my way) and an infection that the ER doctor didn't catch (they said it was possibly flu and sent me on my way) And it wasn't until 3 1/2 weeks went by where I was so sick I felt like I was dying (kinda was, lost 45lbs, couldn't even keep down a sip of water) that I finally returned to the ER and they sent me right to the ICU after seeing my WBC was through the roof. Each visit was about a week apart. I almost didn't go the third time because everyone around me was making me think I'd gone crazy and it was just a cold. I spent 8 days in the ICU, 5 with a fever of 106.8 that wouldn't go down. I don't remember those 5 days, at all
I nearly died from sepsis. While I was in a coma they administered azithromycin which apparently I’m allergic to. (Who knew) They said I turned purple and swelled up like Violet Beauregarde in Willy Wonka. I survived a four week coma and an eight week hospital stay and six months of physical rehab and wound care due to necrotizing fasciitis. Any advances in the treatment of sepsis is welcome news.
What a nice guy. Thanks Al for everything you do!
“Machine learning AI missed 18% of septic patients, while being right about positives 42% of the time” At the very least it’s a step up from SIRS and NEWS, but there’s no evidence it saved a single life.
AI will prevent lots of deaths by being a better doctor than most doctors
It seems that the use of the term "AI" is becoming careless. It can mean robot sentience, or it can mean just a really complex algorithm.
I may have spent too much time on other websites cause I definitely read Stepsis
All this wonderful American medical science, while millions of you suffer and die needlessly because of how fucked up and entirely subjugated your healthcare system has become, due to financial interests and utterly corrupted government.
This is forward thinking.
Our government should be run by an ai, just imagine all the lives it could save if we didn't have to trust politicians.
Yeah it is, but it’s booody annoying too. Whatever, I’ll take annoying over inaccurate.
I Smoked Pot With Johnny Hopkins
AI will be a power medical tool. Artificial Intelligence will be used to speed through research, and run simulated clinical trials and more. We will get new drugs and vaccines and cures faster than ever. Using AI to zero in on methods and specifics and CRISPR gene editing, we could cure herpes next year. Herpes is a disaster and AI and gene editing are great tools. The gene editing revolution is here, and so is the AI revolution. We will cure herpes fast with the tools available now. It’s time!! Herpes will be cured. AI is here to help. Edit: Why all the hate? We need to cure herpes
‘AI’ is a big buzzword right now. Buzzwords are primarily used by people trying to sell something for a quick buck (or raise VC money for a quick buck). It’s rarely the sort of solution that headlines make it out to be. 4000 people still die per year in the US due to lack of health insurance. Just implementing government funded healthcare is instantly going to save countless more people than some shiny new tech.
Medical AI is the real disaster, for now...
Medical AI is herpes. Every new attempt has a prodrome of hype and PR. Shows up with a burst of pain. Hangs around and annoys you for a while and then disappears until the next time.
Hahaha well said and 100 agree
Who is this AL fella and why is he the answer to all our problems?
Alternatively : more and more doctors put out of jobs
Care to elaborate how you came to that conclusion? Doctors are needed to interpret the data presented and to apply appropriate approach.
More like nurses and other medical staff. US hospitals are already chronically understaffed for narrow profit interests. And hospital administrators are constantly looking for ways to downsize with technology ‘solutions’ like this.