Augmentation, not replacement
The tool can operate independently and make adjustments to chemical dosing, pump curves, and blower speeds with a frequency that most operators and managers simply don’t have time to do themselves.
RLCore CTO and Co-founder Martha White said RLTune can take that cognitive load of monitoring and optimizing of certain systems off the operator’s mind to free them up for higher functioning tasks. Using historical data, the tool will indirectly learn what the operators were doing and improve beyond their abilities because there is “much more fine-grain control,” she added.
“People can’t sit at a terminal and change a dosing rate every five minutes. They don’t have the time for that,” White said. “They have other things they have to do. So we still like to think of this like a very focused operator intern that assists by doing this thing that an operator doesn’t have time for anyway. So we’re not really replacing what they do. We’re just augmenting what they do by doing this with fine-grain control.”
Rao and White both noted, of course, the reservations managers and operators have when it comes to trusting systems like what they’ve built. As such, manual control is always at the operator’s fingertips, particularly because operators have contextual knowledge that the tool may not have.
“What we have designed is a workflow where the operator is always in control because we don’t think the world is ready for a completely autonomous AI algorithm,” Rao said. “Let’s say there’s an NFL football game. The agent doesn’t know that. The operator knows. He knows the load will go up and something will happen.”
White said that the AI does not directly learn from the decisions an operator makes when they take over for manual control, rather, it will indirectly learn operator choices. The import of historical data during what Rao called the “warm-up phase” would include operational choices from the plant management and operators simply because they’d have made operational decisions. In that sense, it indirectly learns from those choices.
Current and future applications for reinforcement learning
As noted, the most effective targets for implementation of RL in water and wastewater systems are in high-cost areas like chemical dosing, pumping and aeration control. In the case of chemical dosing, improved optimization could make a serious dent in the cost of chemical purchases. Aeration optimization, Rao said, shows some of the greatest promise or returns on investment for utility executives and directors.
“We just signed with a customer and their power bill is about a quarter million every month,” Rao said. “Our simulation shows we will save double digits: 15% or 20% savings. In the next two months, we’ll have data. Even if we save 5%, we are happy.”
Looking toward the future, both Rao and White see numerous opportunities for RL to improve system performance for water and wastewater operators, including those in industrial and private industry applications such as pulp and paper manufacturing, food and beverage production and chemical manufacturing. So long as there is a continuous feedback loop of information for the reinforcement of the AI, the optimization can be realized. Rao and White also know they need to keep that ambition in check as they scale the business.
“All of them, but one at a time,” Rao said of the industrial water market segment opportunities. “We are a startup. We cannot lose focus, so our focus over the last one to two-and-a-half years was, “Let’s establish a market fit. Let’s make sure what we are building actually means something for our customers.”
