When it comes to caring for underserved patients, health tech is still figuring out what success looks like

For startups and investors aiming to bring health technology to underserved populations, evidence of impact should be everything: It’s what shows employers and payers a new offering is worth paying for, and what can help convince patients to give it a try.

But while the market is crowded with companies claiming their products meaningfully improve health — especially for underserved groups — there is still no industry-wide standard slate of metrics to fully evaluate them. Some are reporting how many new appointments they’ve created among lower income or underrepresented populations; others are coming up with new ways to approximate how their app or wearable specifically helped rural and urban patients with chronic conditions, for instance.

“It’s a really big problem,” said Alex Lennox-Miller, a health care analyst with venture research firm CB Insights. “So much of the future of health systems is to be in care that’s happening outside of appointments — as we’re talking about patient engagement, as we’re talking about remote monitoring, as we’re talking about home care, those are the places we have the worst ways to measure and the least visibility into their impact.”

Measurement challenges extend beyond issues of access and equity; venture fund Rock Health recently developed its own “clinical robustness” score after concluding that only a small portion of health tech companies even make any claims about their clinical impact on patients.

And while there’s growing interest among investors in turning from traditional financial metrics like market size and growth rate to numbers reflecting meaningful clinical impact — lowered A1c levels, for instance —  it’s “still relatively early, so these metrics are not yet fully standardized,” said Sunny Kumar, a partner at GSR Ventures, which invests in early-stage health tech companies.

Still, there’s debate about what exactly companies should measure to show that they’re meaningfully reaching patients who might not have strong established relationships with traditional health care. “What is a clinical touch? Does a text message count, is it nursing visits, is it only when they come into the clinic?” said Zen Chu, who leads MIT’s Hacking Medicine Initiative.

Given that companies are still experimenting with new measures, it’s typically up to health systems or employers buying these services to set their own benchmarks for equity — whether it’s reaching specific populations or getting them to stay active on the app or wearable device, Lenox-Miller said. “It’s almost more of an issue for investors, because ideally if you’re a buyer or health system looking at these solutions, you’re going to have done the legwork ahead of time to figure out the issues you’re trying to address.”

Investors, on the other hand, should “be aware that these aren’t questions with hard answers yet and start paying attention to what companies, what founders, are acknowledging this and saying, ‘We’re figuring this out, we don’t have a solid answer,’ [and] a willingness to even acknowledge that this is a question,” Lenox-Miller added.

If they don’t get it right, he said, companies making unsubstantiated claims about equity risk losing trust from providers and payers aiming to reach those underserved groups. “What they’re doing is trying to fake it till they make it and hoping that they stick many answers out before people start figuring out that they’re really not making a difference. That destroys the trust that providers, health care organizations, and patients have,” he said.

The lack of standards also presents an opportunity for companies to get creative, he added. Ventures like Alphabet-spinout Cityblock, which focuses largely on urban Medicaid populations, “have the privilege to almost be able to start to figure out what some of these metrics should be and what those data points should look like,” he said. Cityblock did not respond to STAT’s request for comment on how it evaluates impact.

Some industry groups are whipping up their own tools for better quantifying health tech companies’ reach into underserved populations. The Digital Medicine Society helped build a “Market Opportunity Model” that CEO Jennifer Goldsack said companies could use to slice their market and revenue size opportunities by variables like age, race and ethnicity, income, education, and disability status.

But she said health tech companies and investors are often preoccupied with measuring factors like whether new patients download or use apps instead of focusing more on whether their health improves. “As an industry we’re drilling down on the wrong thing. We’re spending time with great consternation about new sorts of process measures, engagement, and retention. How about whether it makes people better?” she asked.

Health tech companies facing increasing economic uncertainty and belt-tightening from customers — including employers and payers — could find that clearer metrics on patient impact will make or break their businesses, said Rock Health president Tom Cassels. “What I’d expect to see is a reversion to the mean with respect to pitches, which is, ‘we can’t go in without some evidence of life or patient outcomes, because our competitors have them.’”