By Neil Parker B.Sc., C.Eng., MICE, CIWEM
$7M project for one of the world's largest utility SCADA systems
Sept. 13, 2002 -- For Thames Water, managing the process of water abstraction through to delivering treated water to over 5 million customers in London involves the control of the plant at over 150 locations.
Most of these are unmanned as the London Water Supply (LWS) supervisory control and data acquisition (SCADA) system carries remote control signals to operate the plant and to monitor its status.
The existing system was developed in the late 1980's in parallel with the implementation of the London Water Control Centre (LWCC) to manage the operation of the Thames Water Ring Main (TWRM) with 14 pump out shafts delivering drinking water to 60 service reservoirs.
The LWCC now also manage the 2 area control centres (ACC's) at Hammersmith and Merton along with 12 sub-control centres which collectively control over 200 pumping stations, borehole's & unmanned treatment sites.
It also monitors output from the 5 major surface Water Treatment Works (WTW's) some of the largest in Europe with a combined treatment capacity of 675 million gallons (U.S.) per day.
The requirement to move to a new system was driven by the increasing business need to be able to share data within the SCADA system with other operational and management information systems by capitalising on the emerging philosophy of open system architecture, building on Microsoft application technology already introduced within Thames Water, along with the need to introduce a long term historian database, automatic report generation and an ad-hoc query system.
This was coupled with the performance problems on the existing system in terms of hardware & software redundancy, workstation restrictions and hardware reliability resulting in increasing system down time. In some parts, the system was also reaching database capacity and had limited ability to add new areas to the system.
At the time when the decision was taken to replace the current system, it was also not year 2000 compliant.
The search for a replacement system was based on the principle guidelines that it should improve flexibility, scalability and functionality without compromising system performance. This included rationalising the system hardware and reducing the number of graphics and system alarms.
To meet this it was decided the system real-time database should be built on the concept of object orientated technology (which groups all the otherwise separately listed attributes & components of say a pump into one object into the database) and be able to support open database connectivity (ODBC).
To comply with Thames Water's IT strategy the chosen operating environment was Windows NT with true 32 bit client server architecture installed as a distributed system with full dual redundancy, able to support TCP/IP protocol standards, interface to the three types of existing plant control equipment and be integrated into the existing telecommunications network.
Following completion of the User Requirement Specification and outline design documentation in October 1997 an open advert was placed in the European Journal based on a set of pre-determined qualification criteria.
The 13 companies that replied were then reduced to six for formal tender invitation in January 1998 by eliminating those that did not meet the all the necessary compliance. The contract was to be let under the Thames modified I. Chem. E. Green Book conditions with detailed design and implementation to be carried out by the successful contractor.
The emphasis was not only on getting the right system at the right value but also awarding the contract to a contractor who could demonstrate their commitment to the project and to working as a partner. Following tender evaluation and interviews with the leading two tenderer's the contract was awarded in May 1998 to Aston Dane plc.
The Requirement Specification for the new system soon identified that the ideal product was not readily available in the market. However Aston Dane's approach as an independent Systems Integrator had enabled them to review an number of possible solutions before offering Verano's (formerly part of Hewlett-Packard) RTAP NT as the preferred choice.
The LWS SCADA system was to be the first major RTAP application world-wide running in native NT and it was recognised that there would be a significant amount of development work necessary to achieve the conversion from Unix to NT and provide the required functionality by way of a new Visualiser man/machine interface (MMI).
Initial user workshops were held to define the overall system design and architecture. This led on to the detailed design phase and subsequent completion in November 1998 of a Master Functional Design Specification (MFDS) that set out the standards for the overall system.
For implementation purposes the contract was broken down into project areas with individual Functional Design Specifications (FDSs) for each one to reflect the 5 major WTWs (Kempton, Walton, Ashford Common Hampton & Coppermills), the ACCs and Sub-Control Centres, the Thames Valley Raw Water Abstraction, the TRWM and the LWCC - the nerve centre for the entire operation.
The replacement of the existing system posed many technical challenges not least in the requirement to change over from old to new while keeping the Treatment Works and Distribution network fully operational.
Mapping the existing flat file data into new "objects" is a major undertaking when considering the current system has 300,000 data points!
In order to satisfy the initial tight time constraints for the project, and to meet some of the technical requirements imposed by such a large and complex system, Verano had to accelerate the development road map for RTAP NT.
In parallel Aston Dane concentrated their effort on the detailed design including data mapping, database design, historian configurations, web browser design for reporting and ad-hoc querying, alarm and security coding as well as the Visualiser MMI constructions.
Although the First Customer Shipment (FCS), a fully supported version of native RTAP NT including the Visualiser MMI, was released prior to Christmas 1998, the project team took the decision in January 1999 that overall completion would not be possible in time for the Millennium and the decision was made to implement year 2000 compliance measures on the existing system.
The site installation programme was to be phased on a site by site basis thus progressively changing out the old system for the new and would run in parallel with the existing system to avoid any potential impact on the day to day management of supplying water to London.
Implementation at Kempton WTW commenced late spring 1999 and was completed end of November. However performance and stability problems began to surface despite rigorous factory testing prior to shipping to site. The problems related to new Visualiser interface and system redundancy software causing work- station lock ups and slowing of performance.
The resolution of this became a major re-write of software and re-testing taking many iterations to chase down and eradicate all bugs. To do this took over one year as frustratingly problems encountered at site were often difficult to duplicate in a factory test environment.
Implementation was finally able to progress last Spring starting with the Raw Water Abstraction system followed by Walton WTW last Summer.
Despite some further software interface issues (unrelated to the major problems above) that had to be overcome, positive progress has been made. These three sites are now fully running on Verano's RTAP NT with the old system decommissioned. Many of the benefits of the new system are now being realised and the operators' confidence in the system continues to grow.
Implementation has now finished at Ashford Common, the largest WTW, and is well under way at Hampton WTW. The ACC's, TWRM and LWCC have also commenced installation in parallel with the WTWs.
Coppermills will follow in the early Autumn once Hampton is complete. Due to the sheer size and complexity of the system and the need to undertake acceptance testing in a controlled manner overall completion is now expected by the end of June 2003. Along the way the project team have also had to amend and grow the system to incorporate the new drinking water regulations for cryptosporidium & plumbosolvency and new engineering projects.
Despite the length of the project the system and technology developed is still at the forefront of the SCADA industry. The project team have been able to benefit from the emerging web technology to develop a first class ad-hoc query system that is now being adapted for use elsewhere in the business.
About the Author:
Neil Parker is a Senior Project Manager for Thames Water.
Contact information:
Verano -- Visit URL http://www.verano.com or email [email protected] for more information or help.
Thames Water -- For more information, contact David Holloway, Water Operations SCADA Manager for Thames Water, London Water Control Centre, Lower Sunbury Road, Hampton, Middlesex TW12 2ES, England, email [email protected].
