\n<\/p>\n
Generative AI, which<\/span> can create and analyze images, text, audio, videos and more, is increasingly making its way into healthcare, pushed by both Big Tech firms and startups alike.<\/p>\n Google Cloud, Google\u2019s cloud services and products division, is collaborating with Highmark Health, a Pittsburgh-based nonprofit healthcare company, on generative AI tools designed to personalize the patient intake experience. Amazon\u2019s AWS division says it\u2019s working with unnamed customers on a way to use generative AI to analyze medical databases for \u201csocial determinants of health.\u201d And Microsoft Azure is helping to build a generative AI system for Providence, the not-for-profit healthcare network, to automatically triage messages to care providers sent from patients.\u00a0\u00a0<\/span><\/p>\n Prominent generative AI startups in healthcare include Ambience Healthcare, which is developing a generative AI app for clinicians; Nabla, an ambient AI assistant for practitioners; and Abridge, which creates analytics tools for medical documentation.<\/p>\n The broad enthusiasm for generative AI<\/a> is reflected in the investments in generative AI efforts targeting healthcare. Collectively, generative AI in healthcare startups have raised tens of millions of dollars in venture capital to date, and the vast majority of health investors say that generative AI has significantly influenced<\/a> their investment strategies.<\/p>\n But both professionals and patients are mixed as to whether healthcare-focused generative AI is ready for prime time.<\/p>\n In a recent Deloitte survey<\/a>, only about half (53%) of U.S. consumers said that they thought generative AI could improve healthcare \u2014 for example, by making it more accessible or shortening appointment wait times. Fewer than half said they expected generative AI to make medical care more affordable.<\/p>\n Andrew Borkowski, chief AI officer at the VA Sunshine Healthcare Network, the U.S. Department of Veterans Affairs\u2019 largest health system, doesn\u2019t think that the cynicism is unwarranted. Borkowski warned that generative AI\u2019s deployment could be premature due to its \u201csignificant\u201d limitations \u2014 and the concerns around its efficacy.<\/p>\n \u201cOne of the key issues with generative AI is its inability to handle complex medical queries or emergencies,\u201d he told TechCrunch. \u201cIts finite knowledge base \u2014 that is, the absence of up-to-date clinical information \u2014 and lack of human expertise make it unsuitable for providing comprehensive medical advice or treatment recommendations.\u201d<\/p>\n Several studies suggest there\u2019s credence to those points.<\/p>\n In a paper in the journal JAMA Pediatrics, OpenAI\u2019s generative AI chatbot, ChatGPT, which some healthcare organizations have piloted for limited use cases, was found to make errors<\/a> diagnosing pediatric diseases 83% of the time. And in testing<\/a> OpenAI\u2019s GPT-4 as a diagnostic assistant, physicians at Beth Israel Deaconess Medical Center in Boston observed that the model ranked the wrong diagnosis as its top answer nearly two times out of three.<\/p>\n Today\u2019s generative AI also struggles with medical administrative tasks that are part and parcel of clinicians\u2019 daily workflows. On the MedAlign benchmark to evaluate how well generative AI can perform things like summarizing patient health records and searching across notes, GPT-4 failed in 35% of cases<\/a>.<\/span><\/p>\n OpenAI and many other generative AI vendors warn against relying on their models for medical advice<\/a>. But Borkowski and others say they could do more. \u201cRelying solely on generative AI for healthcare could lead to misdiagnoses, inappropriate treatments or even life-threatening situations,\u201d Borkowski said.<\/p>\n Jan Egger, who leads AI-guided therapies at the University of Duisburg-Essen\u2019s Institute for AI in Medicine, which studies the applications of emerging technology for patient care, shares Borkowski\u2019s concerns. He believes that the only safe way to use generative AI in healthcare currently is under the close, watchful eye of a physician.<\/p>\n \u201cThe results can be completely wrong, and it\u2019s getting harder and harder to maintain awareness of this,\u201d Egger said. \u201cSure, generative AI can be used, for example, for pre-writing discharge letters. But physicians have a responsibility to check it and make the final call.\u201d<\/p>\n One particularly harmful way generative AI in healthcare can get things wrong is by perpetuating stereotypes.<\/p>\n In a 2023 study out of Stanford Medicine, a team of researchers tested ChatGPT and other generative AI\u2013powered chatbots on questions about kidney function, lung capacity and skin thickness. Not only were ChatGPT\u2019s answers frequently wrong, the co-authors found, but also answers included several reinforced long-held untrue beliefs that there are biological differences between Black and white people \u2014 untruths that are known to have led medical providers to misdiagnose health problems.<\/p>\n The irony is, the patients most likely to be discriminated against by generative AI for healthcare are also those most likely to use it.<\/p>\n People who lack healthcare coverage \u2014 people of color, by and large<\/a>, according to a KFF study \u2014 are more willing to try generative AI for things like finding a doctor or mental health support, the Deloitte survey showed. If the AI\u2019s recommendations are marred by bias, it could exacerbate inequalities in treatment.<\/p>\n However, some experts argue that generative AI is improving in this regard.<\/p>\n In a Microsoft study published in late 2023, researchers said\u00a0they achieved 90.2% accuracy<\/a> on four challenging medical benchmarks using GPT-4. Vanilla GPT-4 couldn\u2019t reach this score. But, the researchers say, through prompt engineering \u2014 designing prompts for GPT-4 to produce certain outputs \u2014 they were able to boost the model\u2019s score by up to 16.2 percentage points. (Microsoft, it\u2019s worth noting, is a major investor in OpenAI.)<\/p>\n But asking a chatbot a question isn\u2019t the only thing generative AI is good for. Some researchers say that medical imaging could benefit greatly from the power of generative AI.<\/p>\n In July, a group of scientists unveiled a system called c<\/span>omplementarity-driven deferral to clinical workflow (CoDoC), in a study published in Nature. The system is designed to figure out when medical imaging specialists should rely on AI for diagnoses versus traditional techniques. CoDoC did better than specialists while reducing clinical workflows by 66%, according to the co-authors.\u00a0<\/span><\/p>\nGenerative AI might not be what people want<\/h2>\n
Generative AI can perpetuate stereotypes<\/h2>\n
Beyond chatbots<\/h2>\n