More than three years after it was determined that lead from distribution pipes in Flint, MI, was leaching into drinking water and affecting residents’ health, thousands of homes in the city still have lead pipes. The pipe replacement effort has been complicated by both politics and residents’ mistrust of a machine-learning model designed to help determine which homes had pipes made of lead.
A grave challenge emerged when city administrators launched an excavation and replacement program in 2016. The city’s record system consisted of thousands of old cards used to catalog public infrastructure, making it difficult to determine which homes have lead pipes. Officials soon realized that Flint not only had a lead pipe problem, it also had an information problem.
A group of volunteer computer scientists, led by Jacob Abernethy of Georgia Tech and Eric Schwartz of the University of Michigan, also recognized a prediction problem—a sequential decision-making process was needed to indicate where to dig next, under uncertain conditions. The known variable was that lead pipes were most likely to be found in postwar homes and least likely to be found in newer homes. So the team created a machine-learning model to help narrow down the search for homes that were most likely to have lead pipes. The results of each new dig could be fed back into the model, improving its accuracy.
The artificial intelligence was designed to help city crews dig only where pipes were expected to need replacement in order to expedite the process. And it worked. According to an Atlantic article, workers inspected 8,833 homes, and of those, 6,228 homes had their pipes replaced—a 70% rate of accuracy.
Flint residents complained, however, that their neighbors’ pipes were excavated and replaced while their pipes were left in the ground. It seemed inequitable. Mistrust of the machine-learning model fed this concern. And project managers could not simply tell property owners to trust the AI modeling program with the health of their families at stake.
After the city contracted AECOM, to speed up the project in 2018, citizens began noticing a decline in accuracy. The new contractor had scrapped the AI program and was struggling to locate the sinister pipes efficiently. “As of mid-December 2018, 10,531 properties had been explored and only 1,567 of those digs found lead pipes to replace. That’s a lead-pipe hit rate of just 15 percent, far below the 2017 mark,” reported The Atlantic.
As political pressure mounted, city administrators and Flint Mayor Karen Weaver demanded that the AECOM dig across the city’s neighborhoods and replace the infrastructure of every house on selected blocks, rather than picking out the homes likely to have lead because of age or property type. While the decision has undoubtedly increased the replacement program’s cost exponentially, it may also offer peace of mind to the beleaguered residents of Flint.
Negotiations are also currently underway to incorporate the machine-learning modeling program in AECOM’s future work. But this complex issue raises an interesting question: how confident are we in AI technology? If your family’s health were at stake, as is the case for many Flint residents, would you trust a machine learning model?
