How Codex Helped Talius Improve Their System Stability by Archiving Over 80% of Their Operational Database Using AWS & Databricks

Client Background

Talius is an industry leader in providing technology solutions in the Aged Care sector. They empower care providers to offer better care through data driven insights.

Talius has partnered with Codex to address a number of issues they are experiencing with their operational database. This case study explores how Codex helped Talius to overcome their technical limitations.

Challenge

Before engaging Codex, Talius was experiencing excess load on their server CPU due to the volume of data stored in their database. This was impacting their ability to both complete adequate reporting and to ensure the stability of their system.

Solution

To address these challenges, Codex initially worked to move all older data into Databricks to alleviate the load on the database. This utilised a number of Lambdas to copy data from the operational database into an S3 bucket, which is then ingested into Databricks. Once this was done, Codex then built the ability to regularly move historic data to Databricks automatically.

Implementation

To successfully implement this, Codex started with a two-week engagement which analysed the current state of Talius’ database and provided recommendations for improvement. Following the initial two-week engagement, Codex started a three-month engagement with Talius which aimed to implement the following:

Implement a new Databricks environment to allow for more efficient reporting to be built.

Archive a significant amount of data from Talius’ operational database, which reduces the load on Talius’s operational database.

Set up the capability for Talius to enable a regular automatic archiving process, to prevent excess data from being stored in the operational database.

To achieve these outcomes:

Codex worked with key stakeholders at Talius to identify tables that contained a large amount of historic data.

Codex analyzed how best to transition a large volume of data without impacting the day-to-day functions of Talius.

Codex built out the capability for Talius to complete an initial archive of data from their operational database.

Codex built the capability to regularly archive additional data from the operational database.

Results

As a result of the initial archiving activity, over 600 million rows of data were moved from Talius’ operational database into Databricks as part of the initial archive. All of this data is now available within Databricks.

These outcomes enable the following benefits for Talius:

Preparation to begin working on AI and ML use cases.

The foundations for building a suite of comprehensive reports.

A more stable operational environment.

By implementing an archiving mechanism and building out the foundations of a data platform, Codex has ensured that Talius has a more stable operational database and is set up to better utilise the data that they have available.

Is your company facing similar issues? Contact Codex today to find out more.

Graham Russell

Managing Director

Talius

It has been a pleasure working with Codex, they are true professionals. The project came in on time and budget, I can’t remember the last time I said that.