SEATTLE — The company called One Concern has all the characteristics of a buzzy and promising Silicon Valley start-up: young founders from Stanford, tens of millions of dollars in venture capital and a board with prominent names.
Its particular niche is disaster response. And it markets a way to use artificial intelligence to address one of the most vexing issues facing emergency responders in disasters: figuring out where people need help in time to save them.
That promise to bring new smarts and resources to an anachronistic field has generated excitement. Arizona, Pennsylvania and the World Bank have entered into contracts with One Concern over the past year. New York City and San Jose, Calif., are in talks with the company. And a Japanese city recently became One Concern’s first overseas client.
But when T.J. McDonald, who works for Seattle’s office of emergency management, reviewed a simulated earthquake on the company’s damage prediction platform, he spotted problems. A popular big-box store was grayed out on the web-based map, meaning there was no analysis of the conditions there, and shoppers and workers who might be in danger would not receive immediate help if rescuers relied on One Concern’s results.
“If that Costco collapses in the middle of the day, there’s going to be a lot of people who are hurt,” he said.
The error? The simulation, the company acknowledged, missed many commercial areas because damage calculations relied largely on residential census data.
One Concern has marketed its products as lifesaving tools for emergency responders after earthquakes, floods and, soon, wildfires. But interviews and documents show the company has often exaggerated its tools’ abilities and has kept outside experts from reviewing its methodology. In addition, some product features are available elsewhere at no charge, and data-hungry insurance companies — whose interests can diverge from those of emergency workers — are among One Concern’s biggest investors and customers.
Some critics even suggest that shortcomings in One Concern’s approach could jeopardize lives.
The New York Times spoke with more than three dozen people, including current and former One Concern employees, board members, clients and investors; as well as experts in machine learning, catastrophe modeling and seismology. The Times also reviewed patents, contracts and communications with customers.
Now San Francisco, an early adopter, is ending its contract with the service, in part because of concerns about whether its predictions are trustworthy. “We can’t be cutting-cutting edge without knowing for sure that we can validate data,” said Mary Ellen Carroll, who leads the city’s emergency management department. The expense, she also said, was burdensome.
After Seattle balked at the cost, One Concern found a company, American Family Insurance, to fund the city’s use of the services. It has paid $250,000 since last year, according to One Concern. In return, American Family gets access to predictions and market insights that Peter Gunder, its chief business development officer, said could influence the “design of our insurance products as well as the pricing.”
Mr. McDonald said Seattle had alerted One Concern to the problems it discovered, including the unanalyzed commercial areas and errors in the seismic assessment of buildings. The building where Mr. McDonald works, for example, was made to withstand strong earthquakes, yet in simulations appeared heavily damaged. The model also often reported surprisingly high numbers of destroyed buildings because, Mr. McDonald realized, it mistakenly counted each apartment in a high-rise as a separate structure.
The company then revised its product twice, adding new sources of building data in Seattle, including satellite imagery, and updating its algorithms. That fixed some issues, but introduced others.
The Costco now appeared in the earthquake simulation, but “the entire University of Washington dropped out,” Mr. McDonald said. More troubling, each update produced vastly different damage predictions when simulating the same earthquake. City workers must now revamp nearly completed plans for sheltering earthquake-displaced residents that were developed using the original version, Mr. McDonald said. (Company leaders said that product iteration was common in Silicon Valley and helped customers.)
Barb Graff, Seattle’s emergency management director, said that, despite frustrations, the city would use the service — especially because it cost nothing. “It’s hard to look a gift horse in the mouth,” she said, adding that she viewed the partnership as a pilot project.
Some former workers also voiced misgivings, even while saying they saw promise in One Concern’s approach. Tom Logan, who interned at the company last year and recently completed his Ph.D. in engineering at the University of Michigan, is among nine former workers who spoke about their experiences. Dr. Logan said the salesmanship — such as claiming to estimate damage on each block with 85 percent accuracy within 15 minutes of an earthquake — was misleading, and it was dangerous for cities to rely too heavily on it.
“One of the major harms is the potential to divert attention from people who actually need assistance,” said Dr. Logan, who said his job offer at the company was rescinded after he raised concerns. “There’s a risk that more lives would be lost than what could otherwise happen” if expert experience was also relied upon, he said.
Ivan Porto Carrero, who oversaw a team of engineers at One Concern, said he was fired in June after speaking out against what he viewed as a company culture of dishonesty. He said the usual start-up attitude of “fail fast and try something new” was inappropriate to apply to disaster response because, “If you fail fast, people die.”
Ahmad Wani, 31, one of the company’s founders and its chief executive, said in interviews that he had repeatedly asked cities for better data about their buildings to improve accuracy.
He said he did not think that more people would die if a One Concern product made errors.
“We are in no way ever telling these first responders that we are replacing your decision-making judgment or capability,” he said.
He added that, as the reach of artificial intelligence expands, it remains difficult to convey concepts of uncertainty to officials who often do not have technical backgrounds and who want clear-cut answers.
He acknowledged that his company sometimes promoted products that were not yet available, but only after “we validate the science is going to work,” he said.
“Our mission is not to make money only,” Mr. Wani said, later adding, “We’re trying to save the world.”
An Alluring Premise
Mr. Wani tells a dramatic company origin story to underline his pitch that the world of disaster response is ripe for disruption.
While studying earthquake engineering at Stanford in 2014, he traveled home to India to get engaged in Srinagar. The Kashmir city flooded, with water rising to his waist in his family’s third-story apartment, he said.
The helicopters that arrived appeared to be rescuing people at random. Hundreds died.
Mr. Wani assumed the disorganization was a problem limited to the developing world. But when he returned to Stanford, he learned that after an earthquake in nearby Napa Valley, callers had overloaded the 911 system, frustrating rescuers who were trying to prioritize those in peril.
In classes that fall, Mr. Wani and other graduate students turned the problem into a project. Using crowdsourced ground-shaking reports from previous earthquakes, they trained a computer system to predict the areas of greatest impact so that responders would not have to rely on 911 calls.
The next year — after refining their algorithm with a database of building damage from the Napa earthquake — the students pitched their services to city emergency managers and venture capitalists. “They thought it was magic,” Mr. Wani said.
The company has since made further improvements and garnered $55 million in venture capital funding, a technology pioneer designation from the World Economic Forum, and big names to its investor list and board, including David H. Petraeus, the retired Army general and former C.I.A. director; Craig Fugate, a former head of the Federal Emergency Management Agency; and Judith Rodin, a former president of the Rockefeller Foundation.
Mr. Petraeus said that in deciding to back One Concern, he relied on the advice of experienced venture capitalists and Stanford graduates who had “a reasonable degree of confidence in it,” as well as his understanding of the “big ideas behind big-data analytics” and his impressions of the founders.
Japan’s second-largest property insurer, Sompo, also made a multimillion-dollar investment after an introduction by John Roos, a former United States ambassador to Japan who has a stake in One Concern. Sompo is paying for the start-up’s services in its first Japanese city, Kumamoto.
Officials in San Francisco were among the service’s earliest fans.
“I was totally blown away,” said Anne Kronenberg, the city’s former emergency management director. The city was using a free FEMA product, Hazus, to estimate earthquake damage. She found it technically demanding. One Concern’s product, by contrast, depicted block-by-block damage in a web browser and promised to refine predictions with artificial intelligence as on-the-ground reports were fed back into it.
Ms. Kronenberg persuaded the mayor at the time, Ed Lee, to support buying the services, which cost $148,000 for the first two years. The company’s predictions were to be used to “determine where our resources should be sent, without even going out with structural engineers and building folks,” Ms. Kronenberg said.
But Ms. Kronenberg retired last summer, and her replacement, Ms. Carroll, recently informed One Concern that the city was terminating the relationship, citing numerous problems. Another California customer, Los Angeles, has allowed its contract with One Concern to expire.
Earlier this year, Arizona became the first customer for a new One Concern product, which the company advertises as being able to project “inundation and impact levels up to five days in advance” of a flood. When asked how that was possible, Ben Colombo, the communications director, clarified that it required a five-day weather forecast. “Once we get that data about expected rainfalls, then we can start running our models,” he said.
Other technology companies, including Google and Fathom, are applying machine learning and other analytical techniques to flood forecasting, with the latter publishing results in major scientific journals.
One Concern also plans to begin marketing a wildfire-prediction product soon that it says will show where a blaze is and where it will move, information that will guide firefighting and evacuations.
Less Than Meets the Eye
Scientific researchers expressed doubts about One Concern’s products. The company has not published its results in peer-reviewed journals, meaning its products have not been independently assessed.
Mr. Colombo said One Concern guarded its methods for competitive reasons. “One could be doubting and cynical and say, ‘Look, do I just have to trust you guys?’” he said. “Yes.”
He said company leaders were, however, increasingly engaged in dialogue with academics.
Applying artificial intelligence to earthquake damage prediction entails significant challenges. Computers must train on large amounts of representative data in order to identify complex patterns, but highly destructive earthquakes are relatively rare, and features of the natural and built environment vary.
One Concern surveyed thousands of buildings after recent earthquakes in Indonesia, Mexico and Alaska, for example, but building methods and ground conditions there often differ from those in the company’s West Coast partner cities, experts said. Databases of building records and other inputs can also be outdated, biased or inaccurate.
Mr. Wani has repeatedly characterized his earthquake product as 85 percent accurate within 15 minutes. A public-relations representative hired by the company, Lauren O’Leary, offered a different figure: 78 percent accuracy when tested across three earthquakes in California and Washington.
In an interview, Mr. Wani struggled to explain the meaning and relevancy of the percentages, which refer to how often damage on a block is correctly categorized. “You know, we don’t even call it ‘accuracy’; we call it a ‘key performance indicator,’” he said.
Mr. Wani began again, before settling on this explanation: “If you have to send first responders to respond after the disaster for, let’s say, carrying out urban search and rescue, you’d be at least 78 percent or higher, or at least more than 78 percent accurate for doing that.”
Zachary Chase Lipton, an assistant professor at Carnegie Mellon University who studies machine learning, said the figure was meaningless without greater transparency, including about how the testing was done and whether the system outperformed simple predictors such as a building’s age. “If you just say ‘A.I.’ and are a little bit charismatic you can raise money now,” Dr. Lipton said.
Ralph Archuleta, a leading expert in how earthquakes affect ground motion, was even more dubious in reviewing One Concern’s claims. “Would I buy this product? No, not a chance,” he said.
A Leap of Faith
Dan Ghiorso, the recently retired fire chief of Woodside, Calif., was drawn to One Concern not only by its algorithms. He was taken by the company’s founders and their promise of success.
People who have worked with Mr. Wani say his role is that of a passionate and demanding visionary. Nicole Hu, the company’s 29-year-old chief technology officer, comes across as disarming and earnest, her clients say. And the third founder, Timothy Frank, 38, is an Air Force officer who juggled fatherhood and doctoral studies while starting the company.
According to the origin story Ms. Hu tells, the three were egged on by their professors and, lacking business backgrounds, spent months participating in a Stanford program that helped them refine a pitch and meet investors. Ms. Hu, who, like Mr. Wani, was born in India, speaks affectingly of the challenge of creating a start-up as young immigrants.
“We’ve grown pretty large, but it wasn’t very easy coming to where we are,” she said in a recent lecture at Stanford. “We really think we can enable a disaster-free future.”
In Woodside, a small town south of San Francisco, One Concern’s earthquake simulations were valuable, including showing that the soil beneath an important rescue route would probably liquefy in a major earthquake, Mr. Ghiorso said.
But that information could have been obtained without artificial intelligence — or any commercial product. A map of liquefaction zones, showing the risk to the roads, is available free from the state.
Similarly, One Concern’s earthquake simulations rely on FEMA’s free damage-prediction method known as P58, with calculations performed by another company, Haselton Baker Risk Group.
“They send us inventory information about what buildings are out there; then we run that method and send them predictions,” said Curt Haselton, the group’s chief executive. “It’s not A.I.”
The map’s demographic feature, used to highlight neighborhoods with high numbers of seniors and people with low income, is also widely available.
Mr. Wani said the typical emergency manager would not know how to look for the various data, which the product integrates and improves upon. “You’re now able to create a holistic emergency response plan taking into account all of these variables,” he said.
Ms. Hu, when speaking at Stanford, said the company was “very transparent” about the accuracy of its models. “In fact, that’s what cities love,” she said.
But some officials disagreed. “We would feel more comfortable if they submitted to a third-party review,” said Mr. McDonald, the Seattle official.
Troves of Data
Public safety agencies have tight budgets, and only a few have paid directly for One Concern’s services. The company has begun teaming up with insurers.
Mr. Ghiorso, the former Woodside fire chief, said he worried that One Concern’s analyses might lead the insurance companies to raise rates.
One Concern hopes to expand into capital markets by scoring cities and companies on their level of resilience, much as Moody’s “creates a credit rating, something is investment grade or junk grade,” said Joe Paluska, One Concern’s head of marketing.
The shift in the financial model “felt very deceitful” and left many employees feeling disillusioned, said Karine Ponce, a former executive assistant and office manager.
The company held a town hall with employees in July to discuss how its insurance partnerships promoted its mission of “saving lives and livelihoods,” Mr. Colombo said. “There’s not enough money in this space to truly achieve our goals without involving the private-sector finance.”
Mr. Wani said insurance companies had important roles in disaster mitigation and received predictions only down to the level of a census block, as opposed to public safety agencies, which could request data for specific buildings.
But representatives of the Japanese insurer Sompo said they believed they would receive the same data from One Concern as did partner cities. Using One Concern’s damage predictions, “we can make customized or optimized underwriting per customer or per building,” said Koichi Narasaki, Sompo’s chief digital officer and managing executive officer.
Mr. Wani said Sompo was a special case because the company insures cities against disasters and already has access to the cities' data.
