My Energy

Project Details

Client: Hive (British Gas)

My Energy

My Energy and My Energy Live were Smart Energy Analytics platforms which Hive built and ran for British Gas. This platform took all smart energy meter readings as well as customer and billing data for British Gas customers. Processing this data to provide customers with easy to understand information on their energy consumption and usage.

Over the course of our involvment with the project grew from processing data from 600k meters to 3.5M meters. This was mostly all processed inside PostgreSQL with heavy use of stored functions to process the data. The database grew from 600GB to 7TB over the project lifetime. The overall system was pretty simple, reliable and high performant even considering the size of the PostgreSQL database.

Data Loading

Chris was instrumental in rebuilding the batch data loading systems for the project. Going from a system which routinely failed and took over 24h to load a day's data, to a system which reliably loads millions of records in under 3h. The new custom data loading framework, written in Java 8, took care of parsing and strictly validating incoming CSV and XML files, then using multi-threaded loading into PostgreSQL. The application has since evolved to manage a number of key data flows in the product.

AWS Migration

Over 6 months Chris and Ian led the successful migration of the my energy product from a Data Centre to AWS. This involved migrating a complex, high performance 7TB PostgreSQL database along with the complete application and utility stacks, and containerizing the application on ECS. During this process the system went through a full security review, resulting in significant replatforming as part of the migration. The migration was a success, with 4M users seeing no downtime or loss of data. It was tricky to migrate whilst the team were still developing features.

The Team

My Energy

Kumaran Sivapathasuntharam

Product Manager

An experienced product manager who is customer problem centric and believes in first design principles. I am passionate about identifying customer problems and solving them in a simple, elegant, and intuitive manner.

My Energy

Chris Ellis

PostgreSQL Engineer & Architect

Working on smart energy insights. Transformed data loading and processing systems. Database development and admin. Migrated entire environment from datacentre to AWS, 7TB database, 3.5M meters, and over 4M active users. Implemented DR. Scaled for pending mobile app launch. Team lead and architect for over a year.

My Energy


Devops Engineer