This piece was published in the July 2026 Vega Health newsletter, The Signal. Sign up here.
As a health tech journalist, I received hundreds of weekly pitches from AI vendors that all sounded the same.
I talked to dozens of vendor CEOs and C-suite executives over the years. They all wanted to sell me their vision in hopes I would mirror the same flowery and hype-filled language about AI.
Journalists and healthcare delivery organizations don’t share much in common (I wasn’t saving lives, for example). But we may be bonded as two of the only partiessubject to this unique form of torture. For me, the worst outcome was maybe wasting my time talking to an executive that I wouldn’t end up covering or writing an uncompelling piece because it never got beyond that initial jargon.
For health systems though, making the wrong investment can be the difference between a disease going detected or undetected, relieving or burdening front-line workers, or introducing a cybersecurity risk
Few vendors ever gave me concrete, differentiated reasons to trust them or their products. Sales, marketing, and public relations are not geared towards nuance or failure.
Don’t even get me started on the back-of-the-napkin ROI math that I was supposed to believe and put into print. A theoretical reach of hundreds to thousands of clinicians, multiplied by theoretical time saved, equals millions of dollars in return. Sure.
I — perhaps like others — would often lean on intuition. I gravitated towards founders that seemed to best understand the problems their companies were focused on solving without relying on the press release at hand. I trusted folks who would tell me unflattering information, who relied on my journalistic discretion and trusted me with information.
When I asked health AI companies about the safety or efficacy of their products, few were able to tell me about technical methods for training models or how they were monitoring their AI. In short, they wanted trust with no proof.
I recently spoke with Vega Health advisor Irene Chen, Associate Professor of Biomedical Informatics at UC Berkeley and UC San Francisco, who told me about her experience researching the trustworthiness of AI in healthcare.
“Historical examples have shown that we have been very enamored with the potential of healthcare AI models and have given model vendors a longer leash to prove[things out] ... there haven't been concrete data evaluations, monitoring results that have been released as a result,” Chen told me.
As health systems are trying to develop processes for procurement and ongoing evaluation of AI tools, I often wonder: where is the accountability for vendors in healthcare AI?
Some say it just isn’t the norm. But I think it’s plain wrong that health systems are bending over backwards to test and monitor third party solutions while the vendor focuses on sales and not taking accountability for the value, safety, and efficacy they are contracted to deliver.
Vendors not only fail to give health systems an upfront evaluation of their products, but many won’t objectively monitor how well the solutions are working once they’redeployed, either.
“The current business model is to sell you this good and then to charge you for [a] monthly subscription,” Chen said. These fees are still charged whether or not the product is delivering on its promise.
Before joining Vega Health, I didn’t know how well AI could be monitored, even as someone who primarily wrote about AI. I’m not a model developer, engineer, biomedical informatics researcher, data scientist, or any kind of relevant technology expert. I’m in good company. Few people working in healthcare as providers, administrators, or policy experts are.
“Stepping back when I think about ethical AI, and health specifically, there are a million questions,” Chen said. “There are a thousand and one ways that things can go wrong. But I do know that each and every one of those ways can be monitored.”
AI evaluation science is not widely discussed in business and policy conversations, but in this stage of rapid AI deployment across healthcare, it needs to be. Irene gave the example of ambient scribes, which are synonymous with the use of AI in healthcare. Even that use case, which is widely lauded as useful for clinicians, requires more transparency.
“Ambient scribe [vendors] release numbers but don’t have any sort of open evaluation,” Chen said. “So, you sort of have a ‘trust me bro’ attitude about these kinds of tools, which is honestly infuriating as a researcher because I would love to know.”
“There's not really an incentive structure if the organization has already purchased a tool,” she added.
Chen is working on a project to collect feedback from clinical users and patients about their experiences with the ambient scribe, noting that the scribe company has not solicited this anecdotal evidence in a robust way.
It begs the question: why aren’t these companies offering full transparency?
Chen has ideas for how the rest of the healthcare community can hold AI vendors accountable. One is creating more publicly available benchmarks for AI tools in healthcare, which could be used as a standard ruler for how well the tools work. Another is creating a third-party database of user concerns or requiring that evaluation and monitoring become part of the contracting process between health systems and vendors.
“There’s an opportunity for us to do a lot better here and for either policymakers to step up or hospitals to say this is how we're going to do it, or a culture change, which would hopefully put pressure on model vendors to say, this is what we expect people to do,” Chen said.
Vega Health is changing the status quo about what it means to work with a third-party technology partner by objectively evaluating and monitoring AI models. We help our customers understand how well their AI solutions are working for them and tell them when they aren’t.
That’s one reason I was so drawn to join Vega Health. We are not an AI company demanding unearned trust. We build trust by listening to and solving problems. And we are committed to showing receipts with AI model performance data to customers and vendors.
We establish clear metrics as part of our initial design process. Among them are technical performance, user adoption, clinical or operational results, and return on investment. Contrary to popular belief, evaluation and monitoring doesn’t lead to extended pilots or longer implementation. Because evaluation and monitoring are baked in from the start, they speed time to value for our customers.
Read more about Vega Health’s approach to AI monitoring from our AI Evaluation and Monitoring Lead, Chris Provan.
