Centralising data sources and enhancing integration through the delivery of a cloud-based data management platform

Responsible for maintaining some of the UK’s most used natural water resources and a wide range of heritage buildings across England and Wales, our customer is a highly respected, UK-based charity.

They collaborate with volunteers and communities across England and Wales to create spaces where local people want to spend time.

Pain points

Although the client already had access to an extensive and sophisticated data landscape, the information was spread across many disparate sources, which was creating inconsistencies when it came to data extraction and usage.

Previously, data integration was performed using legacy technologies, but these methods had become outdated as they struggled to meet the evolving requirements that the deployment of modern solutions had created. As such, some teams developed their own separate processes, which led to inconsistencies as there was no longer a centralised data management approach being followed.

The organisation’s desire to integrate data centrally meant they required a consolidated approach that the entire business would commit to. Rather than having data stored across multiple disparate solutions and records becoming duplicated through the use of inconsistent methodologies, the organisation needed to bring its teams together, establishing a single version of the truth through the implementation of a sophisticated data solution.

Once installed, the solution would eliminate inconsistencies with regards to data lineage and reusability, removing a reliance on on-premise servers and technology, which had become costly to run and manage.

“The team at Circyl has really challenged the way in which we think about data and their approach has been a refreshing change for us. We are already noticing the benefits of having consistent data insights, as it supports our day to day thinking and business operations. Our natural resources are precious and every time we make a business decision, we need to make the right one based on the correct information. Circyl have delivered several engagements with us now and we are pleased to have them as a key partner of our organisation.”

Objectives and approach

To address these challenges, Circyl designed a strategy tailored around the centralisation of business information within a single platform. The first phase of the strategy was to create the core elements of the ‘Integration Hub’, along with key integrations that would deliver a usable and cost-effective solution. Given the time-sensitive nature of the project, Circyl understood the need to work quickly, developing a solution that would deliver immediate business value.

It was also important for the project to provide a greater understanding of what a clear and consistent company-wide data management and integration strategy would offer, and how it would become the basis for future phases to be iteratively built upon.

Outcomes and principles

The Integration Hub project delivered a solution based on a core set of design patterns and principles that allowed the business to understand the benefits of adopting a consistent and centralised approach.

Our customer can now load datasets from multiple source systems into a centralised data platform, allowing information to be easily consumed without any data duplication concerns – it also removes the need for any re-keying of information. By establishing a single version of the truth, the firm has improved both productivity and information accuracy, delivering a unified and consistent strategy for data supply. The ‘load once use many’ policy also ensures data consistency across the organisation’s various teams.

Through the implementation of an Integration Hub built with modern cloud-based technologies, the entire organisation now has access to a flexible data management solution that can be developed and grow over time, allowing it to evolve alongside the business.

Technologies utilised

During the project, Circyl leveraged the following technologies:

Tech_webapi

ASP.Net Web API: An extensible framework for building HTTP based services, including RESTful APIs, that can be accessed in different applications on different platforms such as web, windows and mobile.

Tech_datafactory

Azure Data Factory: ADF is Microsoft’s cloud-based integration tool for managing the extraction of data into Azure from a range of sources. The tool allows configurable pipelines to be developed that automate the data refresh process, keeping report data up to date and removing the need for manual effort.

Tech_sqldatabase

Azure SQL Database: Azure SQL Database is Microsoft’s cloud-based service that provides the database platform for the centralised data warehouse. With its range of service tiers that can be scaled up or down as demand dictates, Azure SQL Database provides the ideal combination of reliability and flexibility for an evolving data platform.

Tech_odata

OData: A well-defined protocol for building, querying, and filtering data via a RESTful API. OData is a best practice way to describe the data and the data model for easy interoperability between data sources, applications, services and clients.

Tech_powerbi

Power BI: Power BI is Microsoft’s cloud-based enterprise and self-service analytical solution that provides data modelling and visualisation capabilities across a range of data sources. 

“I have had the pleasure of working with the client for some time and am proud to support them in making a real difference to the community. They are open minded in their approach to business and every engagement is collaborative in nature. As a team, we really look forward to working with them more in the future.”

Roger Peacock
Business Development Manager, Circyl

Want to know more?

Circyl offer free consultations to allow businesses to discuss their productivity issues and ask us in-depth questions about our services and pricing.

Call 03333 209 969 to speak to a member of our team directly or send a message by filling in the contact form here.