So it gives me a great pleasure to be here today to talk about the guiding principles of artificial intelligence in health care We all know that this is the buzz and this is the thing we're hearing everywhere If you're not talking about AI in health care, you're not doing your job. That's kind of where we are So either you love it You hate it or you're skeptical about this and you could be in any bucket and I kind of move between all three phases For AI integration in health care and we'll talk a little bit about I'm not here to change your opinion about AI in health care I'm here to show you how you can think of AI solutions to solve everyday problems that we're having in health care I'm going to go through a real world example of something that I've created using machine learning tools To figure out a solution for the problems I was facing in my patient population So without much delay, let's get started These are my financial disclosures and have no relevance to the current topic today and As I mentioned earlier, I want you all to think for a moment You don't have to tell me your answers, but just think about it are you an early adopter which means that you're right now using AI tools and Integrating that in your current practice or are you a skeptic where you're like? I will believe it when I see it right or are you someone who says nope? This is not my cup of tea I'm gonna stay away as far as possible from AI integration in my practice and you could be anything There's no right or wrong answers here. It's just what you think is what works for you Now moving on to what we're here to talk about and as I mentioned I'm not here to change your opinion about AI in health care, but I'm here to talk about how you can use it now I'm gonna go through first with the history of artificial intelligence where we are today with AI in health care and Talk about general kind of AI tools What do those words mean people talk about LLMs ML like what does that really mean? right and how can you implement AI solutions in health care with a real-world example and then talk about we're in a profession that holds great Responsibility for our patients and it is very important that we don't take this Lightheartedly and put our foot down with regards to responsibility accuracy and scalability of these solutions in our practice Now let's start off first with artificial intelligence and what the history is now it all started with a possibility a possibility by Alan Turing back in 1950 and his idea or the Turing test was to determine if computers were capable of human intelligence Now that was a that was a very weird idea back in the day Imagine someone saying that our computers are capable of human intelligence in the 1950s It's just something that was very futuristic for people and then six years later John McCarthy defined artificial intelligence as a Science and engineering of making intelligent machines now, we've come a long way from there We've really moved fast forward and there was a there was a lull in the in the area of artificial intelligence for a few years we had wars we had a lots of life changing historic events during this period and There was a delay in the integration and the adoption of AI tools, but come 2000s We've seen an explosion and now what we're seeing is the new era of AI Which is the AI in health care and it makes sense because health care is a tough problem. It's a big problem It's an expensive problem and it's a problem that touches every single human being on this planet So it makes sense that why shouldn't we use AI to solve the problems of everyday patients everyday human beings? So let's talk a little bit about basic tools of AI now we hear about ML machine learning Machine learning is all about pattern recognition Machine learning improve improves the experience with provided data sets you feed a data It gives you pattern recognition tells you where the hotspots in the data is and how can you create digital phenotypes of your patients? Now deep learning is taking it the next step forward It's layers of neural networks that allow machines to learn and make decisions there's a little different the machine learning right and then you have natural language processing which NLPs where computers extract written information and can make decisions from that and the next kind of tool is computer vision where it takes Images and makes sense of it and makes decisions from it So if you think of it as physicians as clinicians, we're experienced in recognizing patterns with labs images clinical presentation outcomes Treatment responses and when we don't know the answers, we're able to identify resources. We read we talk we hurt here experts We go to the next person who's an opinion expert in the space and ask for their opinion and collate all that information to really make a Very informed clinical decision on our patient and there's a lot of parallels and how machine learning tools are Developed and I've just highlighted it to understand how our brain works and how many parallels we have in machine learning tools Now if you hear if you think about this concept, it's all about possibilities just where we started in the 1950s If you think that computers can do what our brains can do It's all about opening up possibilities and what we can achieve here now imagine for a moment you had the information and experience from millions of patients and Thousands of experts to assist you to help the patient sitting in front of you Would that make you faster more accurate and more efficient at what you do? Now just think about that thought and I'm gonna walk you through some clinical scenarios and things about how we currently practice in medicine Now we are here right now where we're doing population-based medicine We take hundreds of thousands of patients where we've done studies and we try to apply it to the patient sitting in front of us and a lot of times we all know this in practice that Medicine is a science and an art and that's where the art piece comes into play Where you can't put the science in every patient and expect the same results. So here this is where we are We're doing population-based medicine. We're doing reactive medicine after someone gets diabetes. We're treating them Why should we be doing this? Why can't we go backwards and start from where the problem started? Can we predict based on patient profiles who's at high risk for getting diabetes based on family history BMI, etc, etc and then that patient is given information up front to prevent the progression of pre-diabetes or diabetes or Diabetes profile into someone who has full-blown diabetes Can we make diagnosis easier for us and the patient and with the diagnosis? Can we get information about which drug is going to be most effective for this patient profile and what is the side effect profile? The percentage of you getting an SGLT-2 induced X is this with a GLP-1 RA is X Imagine if our practice included that how beautiful that would look like how much of stress back and forth with the patient Would that eliminate and that's how I want you to think of AI It's a it's a tool that assists you it makes you faster more efficient more accurate And and it helps our patients and brings back time into our visits with patients where we're really doing what we're truly trained for It's talking to the patients listening to them hearing about them and making those much higher level decisions than what a computer system can give us Now endocrinology as a specialty. We're the perfect use case for AI solutions, right? We're data-driven We work with numbers all day I tell this multiple times in clinic you quack like a duck you walk like a duck if the numbers tell me you're not a Duck, you're not a duck and thank God we have we have numbers in endocrinology. Otherwise, we would be rheumatology, right? So we have numbers and it is highly data-driven and we're doing a lot of this So let's think about how we can create a model and I'm gonna deep dive into My practice and I do endocrine toxicities or endocrine side effects in cancer patients for those of you attended my talk yesterday Have heard that this is an emerging space within the field of endocrinology And we're seeing a lot of patients who are suffering with toxicities and side effects throughout the cancer system Now I'm going to talk a little bit about model development validation Implementing AI products and health care and then always thinking about that that cloud in my head and responsible AI I'm always responsible for the decisions a computer system is helping me make so that's something I want you to think about every time you are looking at these kind of tools So I'm gonna help you think about a problem I want you all to take maybe 10 seconds to think about one problem that you have in your current practice that you wish You had better tools to solve and you all have such varied practices. Just think about what you want to solve This is one thing. I absolutely want for my patients, right? So think about that for a minute I'm gonna walk you through a few steps and how you can use or integrate AI tools to maybe perhaps solve your problems, right? So I think about four main tools. The first thing I want to do is like I mentioned What is your problem define your problem? What is the scope? How many patients is it truly affecting? Is it something that you're believing is a problem or is it truly a problem? What does the data say right and then after that? How do you use all these tools that I talked to you about ML deep learning computer vision natural language processing tools? How do you apply that? How can they help you solve that problem? And then how do you integrate that into your system because at the end of the day you can create a beautiful model on paper Right, but if you don't have the right people to implement it, you don't have they don't have skin in the game They don't have boots in the ground that model is going to fail So you really need to get the end-users? Who are the people who are going to implement the model into the room when you're having these discussions about model development? And then the last thing is lifestyle management. You can create these models. You can implement them beautifully But is it achieving the goals that you wanted to achieve which brings you back to the drawing board if you didn't get what you wanted Which oftentimes nine out of ten it actually does happen It's an iterative process right because you were learning as the computer systems learning So you're always going to be in this bucket of iterating your process about okay. This is working. This is not working Let's go back to the drawing board. Let's see what else can work So you really want to think of these tools in that way as your companion? Who's helping you make better choices and better decisions about the problem that you want to fix? Now, let me talk about a real-world example. This is from my own practice. This is my own study I'm going to walk you through each of those steps that I talked about defining your problem. What is the scope? What are your data elements that you're trying to look at and what are the outcomes you're trying to achieve, right? So my I work in the space as I mentioned I work in the space of side effects from cancer patients and I was seeing this I've been doing this for a decade and what I saw was that patients come In either in the ER in the hospital with all these different kinds of toxicities and their treatments are stopped Their treatments are held their treatments are delayed their cancer treatments. I mean and cancer is progressing They're suffering through this process and I always wondered what if we made a more proactive approach to these patients patients are sick They deserve help right away, right? So you need to get in quick and Solve their issues so that they don't progress to a hospitalization So why not move from a reactive approach where what which is what I was practicing to a more proactive approach So that was my problem that I wanted to solve your problem could be very different now I was seeing this in my clinic, but was that really a problem throughout the healthcare system? This is where the scope this is where you need either a programmer or someone who could pull numbers for you and tell you hey You're seeing diabetes with neuropathy. How many patients you truly have with that? Are you seeing Graves disease with X problems Graves? Arbitropathy how many patients you truly have right? So you need someone to crunch these numbers for you and I have the access to those people So we identified in four years They were about 3,500 plus patients who were started on immune checkpoint inhibitors And what we saw was more interesting 34% a third of those patients a little over a third of those patients were actually seen in the ER or the hospital within the first six months of Starting treatment right alarming number and a majority of those were actually coming in the six percent of those were coming in the first three Weeks after their last infusion, so that tells me the problem was big It was beyond my clinic Right and I'm scoping the problem and defining the need and I take that back and say this is why we need to put Resources this is why we need to put money to solve this problem because a hospitalization costs money a hospitalization is suffering So you can quantify that metric So this is how you define your problem the scope of the issue And then the next step you move on is how do you use those tools, right? I mentioned all these tools. How do you apply that to solve the problem on hand? So what I wanted to do was to create a machine learning based clinical decision tool That can predict the likelihood of a hospital admission or ER visit in patients receiving checkpoint inhibitors and assist oncology teams to in clinical decisions So that was the overarching goal now whether or not I can achieve all of this is a different story But that's what I wanted to do, right? So we move forward with performance targets. This is a very busy slide, so I'm gonna break it down and make it simple. What I wanted to predict was hospitalizations, right? And how do we predict that? Now, when I think of this, I'm like looking at, okay, if you wanna predict a hospitalization for someone with adrenal insufficiency, I'm gonna put in symptoms, weight loss of eggs, fatigue, symptoms, all of these things, ACTH, cortisol levels, and that should be a predictor. And what kind of treatments they receive, that should predict to some extent whether or not someone's sick enough to go to the ER, drop in blood pressure, drop in sodium, change in heart rate, change in weight. So, there are data points which are absolutely objective. They're subjective problems that patients can come in and tell you everything, right? And cancer patients are very symptomatic. So, subjective, you can't rely heavily on that, so you're really looking at objective findings. So, I'm just using the use case of adrenal insufficiency, but we had every specialty, rheumatology, neurology, nephrology, cardiology, everyone put in their data elements that were relevant for their toxicities, and we feed it into the computer system. Computer system puts all this information together, collates it, identifies pattern, remember machine learning when I was talking about that, and then learns on that, and then is able to predict with certain accuracy, and then you figure out what is the predictive accuracy of the outcome that you're getting. And what we were looking for is how good is our model to predict hospitalizations? Are we good at predicting hospitalizations before they occur? Now, the next step that I told you is, a beautiful model on paper does not mean it's going to work, right? So, you need end-user integration or end-user feedback when you want to implement that. And I'm an endocrinologist, I work in the cancer space, but I'm not an oncologist, right? So, I'm not the first go-to person for a cancer patient before they get a toxicity. They come to me after they get a side effect, but they don't come to me before, right? Which is where the model works. So, I needed to integrate this within the world of oncology and say, hey, can you implement this model for me? Can you look at the predictive model and look at the outcomes and do something about it? So, that's kind of the overarching kind of view of it, but this is how we went detailed into this. The model basically predicts high-risk, intermediate, low-risk, puts out these risk scores every 24 hours, and this information is sent out to the oncology team, a small group of oncology nurse practitioners. They review the cases clinically. So, for the high and medium-risk patients, they go into the chart, they review the case, and they either escalate or de-escalate the problem depending on the other clinical findings that they find on chart. So, this is how we integrated it. There was great feedback. We collected user information after that. We did focus group interviews for them to see, did this process work well for you? Was it easy to integrate it? How much time did it take for you to review these charts? And this is a very key point, especially with physician, provider, nurse practitioner, nurse's burnout that we're seeing throughout the world right now. AI integration has to be implemented in a way that it is absorbed by these teams in a very seamless format so that they can continue to use it. It's not like you do it for three months and you forget about it. It's something that they can continue to scale over time. Now, the next thing, this is the most important thing that your programmers will be able to generate is the validity and the safety of your model. That means, this is a safe model. We're really not missing out a lot of really sick patients, right? That is a problem that I had when we were building the model. What if we miss very sick patients? That's a problem, right? But there's still a status quo, which is they'll go to the ER, right? So you're not changing practice. If someone's really sick, the next direction, the clinical practice would tell them, go to the ER. But are we really missing something? So safety, equity, especially in a country like United States, we're looking at, did the data that we put in the system, did it include all demographics, right? All ethnic groups. And are we lopsided to one ethnic group, which is why we've learned on a particular ethnic group and we are not able to predict on a certain ethnic group. So equity is very important to be able to, if you feed in garbage, you're gonna get garbage out. So you have to be really careful about what you're feeding the machine learning model, which again, comes into responsible AI. Now, talking about executing the model and collecting end user feedback, right? So this is the data. This is the most exciting part about when you create a model. This is like when you do a translational research and you finally get the results, right? So this was our results. The top 10 things that actually predicted, someone going into the hospital was the first week of their treatment, change in TSH, pulse rate, age makes sense, lots of comorbidities. So I was very excited to see that we're actually, the results make sense clinically to us, right? So this is what you wanna see. You wanna see what are the predictors? How can we, do these make sense? Do these not make sense clinically to you? But then you come to the chopping board and you're like, okay, what is our final outcome? Were we able to predict hospitalizations? We did a pretty good job. The accuracy was good. It was great. But the actionability was low. And let me explain this a little bit to you. The accuracy was good, but when it came to the nurse practitioners and they looked at it, they saw that the patients were either already admitted in the ER or the patients had a visit the next day with the oncology teams or they were already seen between that time point, right? Because oncology visits are once every three to six weeks. So our data entry points are limited to those touch points in which the patient comes into the system. So the accuracy was good, but the actionability was low. So how do you solve that problem? Which means we need data points between two visits between the oncology teams to fill that gap, to be more accurate. And this is where we are going back to the drawing board and adding muscle to our algorithm. So we're adding things like wearable technology, which captures pulse rates, change in activity levels, blood pressure, and now with the integration of the Oura ring with Dexcom, it's gonna come out in 2025. I don't know if you have Oura rings over here, but they're kind of a ring that you wear. It checks all your biometrics and it's gonna be integrated with Stelo, which is the over-the-counter Dexcom CGM that's in the US right now. And that should be a game changer for a lot of people. So that kind of technology integrating that, and we're actually looking at using Oura rings for this as well. And then looking at molecular data and then adding something called ePROs, where patients can put on either a electronic app or something about how you're feeling. Did you lose weight? Are you feeling tired? And there is a very standardized metrics in oncology that measure this very accurately. So we're trying to integrate that between two visits so our model becomes more accurate and more actionable. So this is where you go through the iterative process. So things that you create are not always 100% absolutely the way you wanna go. And that's the process about AI technology and tools. I'm a co-founder of a health tech company. I can tell you every week we are iterating our process. Every week we're looking at how can we make the product better? What are the existing tools we have to solve our medical problem? So you don't never wanna go from, I have these tools, what do I need to solve? That's the wrong way of approaching things. This is how a lot of health tech companies come into play and are no longer in existence because they're like, I got this great technology, where can I apply it in healthcare? We're here as the stewards of healthcare. As physicians, clinicians, we should be the ones that I have this problem, tell me what tools you have to solve my problem. And that's how it should be because we're here to give great patient care, high-quality, accurate, safe patient care, and it should start from there. And then you integrate technologies to solve that problem. Now moving on to, I don't know if some of you have used ChatGPT, Perplex City. I see with a show of hands how many of you have actually used it. Yay, at least about 50% of the room here, fantastic. I use it a lot, I have no shame in saying it. It has made my life absolutely easy. It is still my thoughts, it's still my ideas, but if I need to make an email succinct, if I need to make it more professional, I absolutely use it, and I encourage you to use it if it makes your life easier. And so this is what I did. After I made this deck, I asked ChatGPT, how can we leverage artificial intelligence to improve the study of immune-related adverse events from immune checkpoint inhibitors? So here's what I got, right? Do data analysis, you can find associations between patient characteristics, treatment regimens, which is what we're doing, right? Develop predictive models, and this can assist you in early detection and accurate diagnosis. I'm like, duh, right, we're already doing this, right? But remember, it took a team of machine learning experts, an expert in the field of endocrinology, oncology teams, right, floor nurses, patients to get to this point. We had huge teams with great expertise to give these answers that ChatGPT gave me in less than 30 seconds, right? So it's fascinating, and I absolutely love this kind of technology that makes me be more efficient with my time. I'm not spending time thinking, it's helping me think faster. Now, this brings me out to large language models, right? We're seeing all of this, ChatGPT is a large language model, and these models have become more efficient, more fast, more scalable, and most importantly, cheap. When we started doing machine learning models four years ago we were spending a lot of money, I mean, 40, 50,000 just for the programming. Now, these things come about as you can use an hour of ChatGPT and it costs you a few cents. I mean, it's incredible how the prices have come down because of the competition. So you're gonna see a lot of this coming in. And what large language models are nothing but next word predictors, which means you put a bag of words together and you put them in a line, and it basically predicts with this combination of words, what is the next word that's gonna go through? So for example, this is a room of endocrinology trained or endocrinologists or people who practice a little bit of endocrinology. So if I tell you that a patient presenting with metabolic acidosis, high blood sugars, positive ketones, what is a likely diagnosis? You all know this, right? Because you're doing the next word prediction. You're like, this combination of this combination of this combination is equal to this. That's exactly how large language models work. That's exactly what they do. They generate content, they translate. You use chatbots. If you've done Amazon, you've talked to airlines, they're all chatbots. They're all computer systems that are talking to you, right? Unless you keep hitting zero and say, I really wanna talk to a human being, you're still talking to a computer system. They can write notes, which we're seeing right now in the US, we have started this ambient technology. They listen to the conversation with the patients and they type out your note for you. So this is happening right now in the US. I don't know how much of this is integrated over here, but we're seeing this more and more in our systems. And e-learning, which means that if someone learns a certain way, they say, I've learned till 10th grade, this is how I learn best visually, or I'm a listener. It will give you personalized learning information to just tailor to how you learn, rather than putting you a one size fits all communication where, hey, here's a bunch of like 25 pages with verbal diarrhea on it and go process it. It's not gonna be like that. It's gonna be tailored to your needs. So which brings me to now. So I've talked a little bit about what AI is, what the promise looks like. I mean, the promise of healthcare and AI is here. The actual integration that's happening is here to the people who use this and know about AI technology. But still, it is better than the status quo. I can promise you that, that wherever we are, this is not a sustainable way of practicing medicine. And AI is gonna be helpful, but you should understand that the promises here, what people are talking, the reality is somewhere over here. It will take some time to get there. And I think our generation of physicians and doctors and trainees is going to spend a lot of time training AI tools. But it's important that we wear the hat of responsible AI, where we're looking at privacy, validity, transparency, preventing bias and discrimination, economic regulation and respect for autonomy. Respect for autonomy, I'm gonna say that twice because we're trained physicians. We're trained MDs, we're trained APPs, and we like autonomy. We want to be the person, the quarterback, either agreeing or disagreeing with the AI-generated clinical decision. We shouldn't allow AI to take over. So I think physician autonomy and provider autonomy is very important. Now with that, I'm gonna end. This is a futuristic slide, but I know my next speaker is going to talk a lot about this. So I'm going to let him walk you through this. And I think I mentioned a little bit about it. And one last thing I wanna say is the sniff test for AI tools in your practice. You're gonna see a lot of AI tools that come through your desk, right? And especially if you're in healthcare policy, if you're in administration in the hospital, this is going to be a reality, either today or in the next few years. Look at, is this AI tool helping me with patient, improving patient outcomes, scale, efficiency, improving my clinical decision-making. So with that, I'm gonna end with one thing. AI will not replace us, but the doctors who use AI will replace the doctors who don't. And this may feel like, wow, this is a bolt, but this is happening. And the sooner we think about this and integrate this into our practices, the better off we are. Well, thank you so much for your time.

artificial intelligence

healthcare

machine learning

ethical considerations

predictive models

proactive care