The Challenge

As one of the UK’s leading supermarkets, this retail client recognised that, in a volatile and highly competitive market, sales reporting is a critical part of its infrastructure. Insight into what is selling in stores is critical for its supply chain and manufacturing operations.

Sales reports on millions of baskets are sent to a central data hub for analysis: previously, even though reports were delivered every 15 minutes, the company’s on-prem batch-processing solution only presented analytics data for the previous day. Having data access in close to real-time would make the company’s stores, warehouses and suppliers more responsive to demand, meaning fewer empty shelves in stores, smarter pricing, better merchandising, more fresh goods and less waste.

In addition, the legacy sales service had developed organically over time, bringing elements of risk into the process, such as inadequate documentation that made significant changes difficult to implement. The retailer required a robust system to feed business-critical data into the company’s general ledger. The retailer therefore needed a more extensible platform that could accommodate changing business needs with the resilience necessary to support mission-critical operations.

The Solution

Working with Appsbroker, the retailer created a new system using Google Cloud Platform tools to enable global sales analysis in near real-time. Reference data such as product information, item descriptions and other details and maintained in Google Bigtable to provide additional performance and responsiveness. The data is then sliced, analysed and extracted at any time of day using BigQuery. With access to virtually unlimited compute power on demand, the system is democratised, so that new queries can be accommodated without any concerns about ongoing performance.

Appsbroker worked with the retailer to address some significant challenges in terms of the complexity of what was required: a large set of business rules are applied to the data, Appsbroker set up Bigtable to hold them as a reference and apply them to the data as it goes through Dataflow.. The new configuration was rolled out across all stores, running in parallel with its old system prior to switchover.

The Results

The key benefit to the client was being able to see their sales data, in close to real time, with a target of making data available for analysis in BigQuery within two minutes of landing in the system. Also, given the sheer quantity of data available, GCP offers the flexibility to tune the system for optimal price-performance. The use of Google’s cloud infrastructure has also meant that datacenter infrastructure and maintenance costs have been replaced by cloud-based, fully managed solutions and that the retailer has been provided with a solution built for extensibility that will scale with demand.

Such a highly available system could be vulnerable to peaks in demand, such as those around the Christmas period. Previously, the team would resize in advance of expected increases in demand, purchasing additional infrastructure that would be redundant most of the time. Now, simple VM management, autoscaling and pay-as-you-go pricing on GCP have delivered a cost-efficient, reliable infrastructure. And because the system is fully documented, built on open standards and designed from the ground up to accommodate change, it can handle whatever the future brings.