How Codex Helped an ASX-50 Infrastructure Company Lay Its Modern Data Engineering Foundations

About the client

A leading ASX-50 Energy & Infrastructure provider in Australia, which owns and manages a portfolio of energy assets worth around $26 billion.

Modernising Data Platforms to Drive Scalable Reporting and AI Innovation

Our client was operating on a legacy, on-premises SQL Server solution for reporting and analytics, which was quickly reaching its capacity limits. Adding new data to the platform had become a cumbersome process, leading to inefficiencies. As a workaround, reporting was increasingly being handled directly within source systems, placing performance and governance burdens on the source infrastructure and operational teams.

Recognising the need for a future-ready solution, the client set out to modernise their data platform to better support evolving business needs. Their goal was to enable self-service reporting, advanced analytics, and AI-driven insights to unlock new business opportunities. The new platform needed to ingest and transform data rapidly, while maintaining scalability, security, and governance.

In tandem with their data platform modernisation, the client was also upgrading its ERP landscape to Workday and Maximo. Critical financial and procurement reporting use cases from these new systems were prioritised as foundational components of the modern data platform, ensuring the business could achieve seamless reporting and analytics across its new operational environment.

Challenge

Robust data engineering foundations had to be put in place for the new data platform, before delivery of any ERP use cases could commence.
These foundations were required to be “enterprise-grade” to support all future streams of work on the platform.
Data engineering approach needed to accommodate for the rapid changes in the new ERP source (Workday), not yet in production.
Solution needed a scalable way to ingest information from Workday, as it didn’t expose its data repositories directly.
Time constraints required delivery of several analytical assets (facts and dimensions) to support the critical Day-1 reporting needs, in line with production deployment of the Workday system.
There were specific security requirements for the analytics solution to mimic the user data access setup in Workday.

Solution

To address the main problem of creating a unified data platform, Codex built the data platform using Databricks with AWS.

As a part of this solution, Codex was responsible for the design and data solution architecture, identifying the data sources, designing the ingestion strategy, designing data models and building data pipelines to deliver a unified data platform, facilitating enterprise wide reporting needs.

Implementation

Identified the data sources and designed the ingestion strategy for structured data using metadata-driven streaming pipelines.

Designed the ingestion strategy from WORKDAY to AWS S3 staging location via CDATA connector in unstructured XML format, followed by using Databricks to parse and store the data in a structured relational format.

Designed the medallion architecture and built data pipelines to ingest data into the Bronze layer.

Designed data pipelines to cleanse and load into Silver tables while maintaining history, which is used in the reporting layer (Gold) and can be leveraged for predictive analytics.

Based on identified use cases, designed the dimensional data model and built FACTS and DIMENSIONS in the Gold layer to meet business reporting needs.

Designed and developed role-based access controls along with row-level security and column masking functionality on the Gold layer, mimicking WORKDAY security and row-level access for end users.

Assisted business users in efficiently designing Power BI models and dashboards.

Results

Successful in ingesting 400 tables into Bronze & Silver across different sources.

Built 30 Dimensions and 15 FACTs facilitating 3 use cases.

A dynamic security solution which can handle object level and row level security based on user’s access in the ERP system (Workday).

Codex addressed the challenge of creating a unified data platform by building a comprehensive solution using Databricks on AWS. The team successfully designed the data architecture, identified data sources, developed an ingestion strategy, created data models, and built data pipelines to enable enterprise-wide reporting capabilities.

Is your company facing similar issues? Contact Codex today to find out more.