by William Spencer
The Orange County Sanitation District (OCSD) operates the third largest wastewater agency west of the Mississippi River. OCSD facilities include 580 miles of sewer pipes that snake throughout the county, and two treatment plants: one in Fountain Valley and the other in Huntington Beach. For more than 50 years, the agency has collected, treated and disposed of, or reclaimed, the wastewater generated by 2.5 million people living and working in central and northwestern Orange County at the rate of approximately 250 million gallons per day.
The district has embarked on an 18-year, $2.5 billion capital improvement plan that will update and rehabilitate many of the facets of the two plants and outlying facilities. The plan includes an immediate $450 million initiative to build a new headworks to screen out solids and a $150 million secondary treatment project that will treat effluent being pumped into the Pacific Ocean. To accommodate the ongoing upgrades and ensure that the plants continue to deliver a high level of service going forward, district leaders decided they needed to create a better system of managing information about their critical infrastructure.
Although more than 600 employees work for the district, the waste screening and treatment process is driven by computers. Each of the plants has two control rooms, roughly 30 by 40 feet, with two VAX servers and a half dozen monitors. The servers run the treatment equipment in the plant as well as the office functions. Although each plant has about 10 stations where operators can make adjustments to the processes, periodically changing temperatures and regulating flows, the plants generally operate on an automated basis.
“People really have no idea how advanced all of these processes are,” said John Stephens, the information technology analyst at the OCSD. “They just flush their toilets and think it all just goes away. We have a lot of technology in here.”
Despite the district system’s high level of automation, the OCSD facilities staff still relied upon spreadsheets and CAD drawings to track all the tube cables, fiber optic cables and wired connections that serve the plants. Many of the documents included incorrect information; others couldn’t be accessed or were hard to follow.
“We really didn’t have a system,” said Stephens. “We had multiple spreadsheets and CAD drawings that tracked different things and overlapped with each other. Whenever a change was made on one, it wasn’t necessarily made on another. There were mistakes throughout, and many of the documents were outdated. We knew problems were going to crop up, so it made sense to start with a whole new system.”
The OCSD turned to Planet® IRM, a centralized infrastructure data management system provided by Planet Associates Inc. (Neptune, NJ). Planet IRM is a visual, consolidated Configuration Management Database (CMDB) for documenting physical critical infrastructure elements and their associated dependencies. It provides total visualization of all network elements (intelligent and passive) and their interconnectivity through geo-referenced, layered campus maps, floor plans and infrastructure space diagrams, and facilitates correct business processes for change management and maintenance activities.
As part of the implementation, Planet Associates personnel first evaluated the OCSD’s current practices and made recommendations for development of change management procedures to be incorporated into the ongoing processes. Established process management is crucial for maintaining an accurate data repository, so it was critical to understand the OCSD’s application goals and develop appropriate procedures early on. The OCSD’s primary goal was to use the database to consolidate all existing information about the plants’ infrastructure technology that can be accessed and updated systematically as new facets of the capital improvement plan take hold.
The next aspects involved determining the types and sources of data that were to be collected and stored; identifying the most efficient method for collecting and configuring data to comply with Planet’s data import formats and the client’s desired management and analytical report outputs; surveying all intended assets to ensure complete and accurate information; developing Product and Cable Library entries to create a definitive list of all hardware and cable objects; and then entering the collected data into the Planet IRM repository.
To ease the population of data into the new system, Planet’s Dynamic Data Reconciliation (DDR) module was used to import the countless disparate spreadsheets of assets and cable information. Additional data from AutoCAD drawings was also imported to establish individual sites and workspaces that could be best used and managed by OCSD personnel.
Before handover, Planet personnel reviewed the change management procedures previously put in place; assisted OCSD personnel in determining the roles and responsibilities necessary to maintain a successful implementation; and provided training services as required.
Prior to implementing Planet IRM, the OCSD had drawings of all its fiber optic pathways, but they weren’t clearly marked. Fiber optics provide the communications backbone of the OCSD plants, controlling the telephone system, the fire monitoring system and public address functions as well as all of the waste treatment processes. The two plants together have about 300 tube cables, each carrying between 6 and 18 strands of fiber. Before, each cable would be pictured on a drawing, but plant operators could not check the service functions or delivery status of each fiber. The new system identifies each pathway through the tube cables, from switch box to switch box, denoting which process each fiber governs, giving operators the ability to access this information, run diagnostics on the lines and produce service delivery reports, all in a matter of minutes.
Essentially, the move to Planet IRM instituted a new set of processes that didn’t exist before. Perhaps the biggest improvement has been with capacity planning, Stephens noted.
“As we’re adding new processes, we can run quick checks on how lines will connect to the control center,” Stephens said. “Do I have pathways to take new fiber from the construction zone to the control center? Before, we’d have to guess. Now we can tell just by looking at the program.”
The system also indicates where dark fiber is available for use, so there’s no more guesswork as to when new fiber runs are required or existing runs can be put into service.
Stephens said the district expects to integrate other Planet IRM capabilities, such as using the tool to troubleshoot problems in the lines and run bandwidth utilization reports, in the near future.
“Our first goal has been to come up with a solution to our problems tracking fiber,” Stephens said. “As we move ahead with all of our projects, we will begin to take advantage of all this tool has to offer.”
About the Author:
William Spencer is CEO, president and founder of Planet Associates. He holds a Bachelor of Science cum laude from Farleigh Dickenson University, and frequently speaks and writes about critical infrastructure and physical network management.
