Employing over 1,400 people, Affinity Water supplies over 900 million litres of water each day to 3.6 million domestic and business customers in three English regions. The largest, Central, extends from Bedfordshire and Buckinghamshire into Hertfordshire, Greater London, Essex and Surrey. The smaller East region covers parts of Essex, while the Southeast region takes in coastal areas of Kent. Key activities focus on the abstraction, treatment, delivery and billing of water services.

“We’re part of the critical national infrastructure: our product is essential to life,” says David Clifton, Enterprise Solutions Architect at Affinity Water. “The key issue is maintaining supplies in the most effective ways. There are many challenges in this sector that we can only overcome using data collection and analysis to improve our knowledge and understanding. We operate in the Southeast of England, the UK’s driest region, an area the Government has designated as being under serious water stress. We must deal with a rising population of high water users and data and analytics can help us ensure there will be enough water for our customers’ needs in the future.” This is where the combination of current expertise and technology such as Amazon Web Services (AWS), Talend come in.

“The UK water industry is heavily regulated,” David continues. “We are obligated to submit a business plan to our regulators in five-year cycles on how we intend to invest and manage those assets so we can ensure the delivery of clean water to our customers at the best possible costs. This Asset Maintenance Plan includes commitments to regulators and customers. To derive it, we need to analyse and model assets in various ways. For example, hydraulic modelling of the pressure required to feed water to properties with growing demand. Models to examine bursts – looking at our pipe network, different pipework ages, materials and wet or dry environments to try and reduce unplanned outages and customer interruptions. Analysing all that data is essential to how we operate, so we can understand our customers better and provide the best possible service to them.”

AWS and Talend

“We made a strategic decision to move from traditional IT delivery to the cloud,” David says, “AWS provides the ability to provision infrastructure services without the traditional upfront high capital investment and lead times.” AWS was selected following a competitive tender: “Its services and cost model are hard to beat, at a time when it’s increasingly difficult for individual IT people to stay abreast of all the new services and opportunities. Our future business operations will depend on the cloud: the ability to analyse data at any scale, with a few mouse-clicks, gives us unparalleled advantage that we could never be achieved using a traditional self-hosting IT model.”

He says it was also essential to standardise data integrations: “to remove swivel-chair manual integrations and legacy script-based file transfers.” For this, Talend was selected, initially to provide more batch-centric integration, although this is changing: “We’re also exploring the Data Quality side of Talend, not least in light of future Data Governance programmes and impending cyber and data security commitments such as GDPR. Master Data Management will become one of our most important future initiatives.”

Why Datalytyx?

With Talend recommending Datalytyx, David says the Datalytyx team “brought the right skills in Talend and other applications such as SharePoint and K2. Datalytyx provide a ‘product support wrapper’, allowing us to trade-off in-house skills.” The largest Datalytyx project to date has been developing a new centralised meter reading management solution. Previously, information was held in a variety of systems including asset management, meter reading, billing and web portals.

The solution collects data from multiple sources, including Internet-connected smart-meters, loading it into a data warehouse structure for ease of management and analytics. New data sources can be added easily. “It’s early days but the solution already allows Affinity Water to capture this data in one place, and is enabling us to meet our commitments to the new open market in the commercial customer space. A recent project Datalytyx helped deliver focused on helping the company trade with other supply businesses efficiently for the fast and accurate settlement of bills. Going forward, this is the kernel repository that will allow us to analyse and understand the demands of customers. With automated meter reading and smart metering we can start understanding real-time events on our delivery network, to trace and fix leaks faster and reduce customer bills.”

Benefits on tap

David adds, “Without this underlying technology we’d struggle to operate and meet our business commitments, and risk financial penalties from regulators and other areas, including requirements such as the EU General Data Protection Regulation (GDPR). We’re certainly seeing gains from AWS and Talend. It’s a little early to assess those in monetary terms, but we’re certainly seeing benefits in lead times to deliver change.” In telemetry, for example, data was previously analysed and reported from an old RDBMS, which wasn’t optimised. “Using Talend and AWS Redshift means our business teams can now execute reports in minutes rather than hours or days. We had some reports we couldn’t execute at all, due to the size of the database; these now run in less than an hour. With AWS Redshift provided ‘as a service’, a large amount of the IT operations needed to maintain underlying systems is taken care of, whilst we also have disaster recovery and restore options. Without the Talend Data Integration product, transmitting the data would be expensive and complex.”

In terms of future, David says it’s all about “Data, data and data. By that, I mean looking at mastering our data and analysing it, so that we can increase our operational efficiencies and reduce our expenditure. Data is a key ingredient for understanding, innovation and automation that will let us better understand demand, develop our real-time operations and situational awareness, and improve customer experience.”