Upgrades, Operational Enhancements Ready Plant for Remote Operations
Like many water departments across the United States, the Haverhill, MA, Water Department was having trouble recruiting and retraining operators essential for its treatment plant operations.
By John D’Aoust and Carol Harris
Like many water departments across the United States, the Haverhill, MA, Water Department was having trouble recruiting and retraining operators essential for its treatment plant operations. Coupled with decreasing operational budgets and increasing operational costs, the department realized the need to improve operations and optimize processes in order to operate in an efficient, cost-effective manner.
Haverhill’s Water Department supplies water to over 60,000 residents and businesses in the community, which is located in northeastern Massachusetts. On average, Haverhill’s 12 mgd water treatment plant produces over 2 billion gallons of water each year. Treatment includes coagulation, tapered flocculation, sedimentation, dual media filtration, granular activated carbon adsorption, fluoridation, corrosion control using zinc orthophosphate, and chlorine disinfection.
In 2001, the city received approval from the Massachusetts Department of Environmental Protection (MassDEP) to revise its operational procedures and conduct remote, automated operations during a portion of the 24/7 plant operations. The MassDEP approval included a number of conditions, including plant upgrades, procedural and reporting changes, and operator training to address security and regulatory compliance.
Haverhill’s updated SCADA system includes expanded alarming capabilities to meet MassDEP requirements for remote operations. Pictured are John D’Aoust and Carol Harris.
To address these needs, Haverhill engaged Woodard & Curran, an integrated engineering, science, and operations firm, to evaluate the situation and develop a solution. Its engineers, O&M experts, and control systems specialists evaluated Haverhill’s problem holistically and optimized the facility and operations for remote operations, with an eye toward increasing efficiency and reducing operating costs.
Remote operations required a number of control system upgrades to help operators recognize and respond to problems from any personal computer. In addition, process and procedural needs were identified to ensure reliability of plant operations.
Upgrades and enhancements to prepare the plant for this new mode of operation included:
- Improving critical treatment process monitoring, instrumentation, and treatment controls.
- Enhancing the plant’s supervisory control and data acquisition (SCADA) system.
- Providing radio communications to remote sites.
- Developing a remote paging alarm system.
- Training staff to understand Haverhill’s new operations and processes.
- Developing site-specific Standard Operating Procedures.
Haverhill required updated, reliable process monitoring equipment, coupled with its SCADA system, to control critical process variables in order to implement remote operations. Instruments updated included:
- liquid level sensors for the chemical containment floor areas,
- a streaming current monitor at each of the rapid mix effluent chambers for measuring and controlling coagulant addition,
- an online fluoride analyzer to measure the clearwell fluoride concentration,
- a new clearwell influent inflow insert magnetic flow meter,
- an ultrasonic level transducer to replace the clearwell float type level transducer,
- an energy management system to monitor the building’s ambient air temperature and HVAC system, and
- a power monitoring device.
Installation of the streaming current monitor was a critical component of the plant’s upgrades in order to detect potential coagulant over feeding or under feeding. In addition to monitoring, Haverhill realized more efficient coagulant feed by programming the coagulant feed pumps to trim off the streaming current monitors.
Testing of Haverhill’s existing SCADA programming identified a number of points for enhancement, both to improve overall reliability and to ready the plant for remote operations. Testing included checking the process monitoring alarms and interlocks to ensure reliable operations; a review of the SCADA system’s ladder logic to ensure functionality; and simulations, through the human-machine interface (HMI) station, of alarm and operational conditions to ensure the programming functioned properly, both inside the plant and remotely.
For the new automated mode of operations to occur, the MassDEP required that the SCADA system be able to remotely monitor, automatically shut down, and call out an alarm for a number of conditions: chemical feed pump failures, pH levels out of range, turbidity exceedences, high or low chlorine residuals, high or low chemical storage tank levels, filter malfunctions, distribution system high pressure, fire and intrusion, power failure, critical pump and motor failures, and remote pumping station failures.
As Haverhill’s existing alarming capabilities were limited, expansion of the plant’s alarming capabilities, and remote access to them, were essential to meet the regulatory requirements. Operators monitoring the plant needed to receive accurate, timely information to effectively assess and respond to conditions at the plant.
The plant’s alarming system was expanded by installing WIN-911, a new alarm software with unlimited alarming capabilities, to replace the existing outdated alarm dialer, which was unable to call out all the alarms required by the MassDEP. The old alarm dialer was retained as a backup system.
WIN-911 is a real-time alarm notification software that works with Haverhill’s SCADA system to monitor operations and notify personnel of problems or alarms.
Operators have secure access to Haverhill’s control system, including alarms, through a VPN connection to the city’s network.
Haverhill’s SCADA system operated on leased telephone lines and had experienced numerous communications failures, causing unnecessary plant shutdowns. The phone lines were replaced with more reliable radio communications between the water treatment plant and the remote sites.
Haverhill was required to obtain a radio license from the Federal Communications Commission. A radio path survey was performed, with radio frequency modeling and field testing to determine the best locations for the SCADA system radios. The radio system is designed to be expandable; additional radios can be added to bring other components of the water system into the network. Since completion of the core telemetry system the city has added a second radio repeater site and three more remote facilities to the radio network.
Process monitoring upgrades included replacing a clearwell float-type level transducer with an ultrasonic level transducer for accurate water level monitoring.
The facility’s Operations and Maintenance Manual dated to when the plant was built in 1980 and did not address the plant’s new mode of operations. A more useful, easily updatable series of standard operating procedures were developed to supplement the existing O&M manual. To meet MassDEP operational requirements, SOPs were also completed for plant shut-down and restart.
Staff were trained on the various topics covered by the SOPs, which included modules on manual plant startup and shutdown, chemical deliveries, chemical transfer, and batching. In addition, the SOPs address routine procedures for operators to follow prior to and after transferring to remote operations and responding to alarms.
Remote operations required increased security at Haverhill’s water treatment facility. A number of physical improvements were made to make the plant itself more secure. A video surveillance system was installed and integrated into the HMI system, and improvements were made to secure various access points to the plant.
Haverhill is in the process of updating its emergency response plan and plans to conduct staff training on the new emergency procedures using tabletop exercises to simulate emergency situations. The emergency response plan maps out the roles and responsibilities of water system staff during emergency operations. It details emergency declarations and levels and includes emergency response procedures covering a host of conditions from water contamination and extreme weather conditions to acts of bioterrorism.
The project commenced in 2007 and a three-phased plan of upgrades, with essential items spread out over the course of a year and a third phase of long-term tasks, was developed. The plan was organized around applying for and receiving MassDEP final approval for remote operations and separated tasks by priority.
Phase I prepared the plant for remote operations by securing the operating data necessary to develop the procedures and upgrades for the new mode of operations. Regulators were informed of Haverhill’s approach to obtain their support of the process. Operational changes readily achievable with plant staff were also implemented as part of Phase I.
In Phase II, final MassDEP approval for remote operations was secured, and key modifications and procedures to ensure that continuous and safe water supply would be supplied to its customers were completed. Updating the alarm system and security enhancements were an important part of this phase.
Phase III, which commenced during the transition to remote operations, addresses long-term support from equipment and operations and maintenance optimization perspectives. It includes an energy audit, implementing energy saving measures, and continuing to improve plant efficiency.
Shifting From Reactive To Proactive
The optimized Haverhill Water Treatment Plant received approval from the MassDEP, successfully operates remotely a portion of the time, and staff tasks have been reallocated to meet operational needs. Haverhill was able to switch from a reactive position to a proactive one. Operations are much more efficient, with plans for increased efficiency in the future, and the staff is able to work on preventive maintenance tasks. WW
About the Authors:
John D’Aoust is Water Treatment Plant Manager in Haverhill. Carol Harris is a Vice President in Woodard & Curran’s Andover, MA, office. She is Former Executive Director of the Massachusetts Water Works Association.
Circle No. 209 on Reader Service Card