A major healthcare marketplace and IT provider wanted to implement offline backup and recovery processes for their Snowflake instance and some AWS services. They also needed an automated way to configure client resources based on the size and tier of the client.
To establish an automated backup and recovery process, phData developed customized solutions within AWS and Snowflake. The expert team also refactored Terraform to create a CI/CD process for the pipeline.Â
A leading online marketplace for finding healthcare providers needed a consistent and cost-effective way to backup and recover its data. With a service that leverages advanced analytics of patient and provider data to build a connected consumer experience, the customer had both volume and certification requirements to consider.
The customer wanted to implement offline backup and recovery processes for their Snowflake instance and some AWS services. They also needed an automated way to configure client resources based on the size and what products a client had purchased.Â
As a solution, phData set up data pushes for backup, allowing backup processes to run as automated processes. The phData team also refactored internal data workflows in Terraform, creating a pipeline using CI/CD processes for increased automation.Â
The customer had previously worked with phData to identify cost-effective modifications to their platform, allowing a three-tier pricing plan. The successful completion of this assessment project was the impetus for adding a project team to implement a portion of the proposed changes, including automated backup and recovery processes.Â
The phData project team pushed using automation to make their processes easier to run and developed several solutions on AWS and Snowflake that allowed the backup processes to roll out as an automation.Â
The team also refactored some of the customer’s internal Terraform repositories to be easier to automate, which we built a pipeline to deploy out those resources with a CI/CD process.
The phData project team implemented backup and recovery processes across three areas:
Snowflake Offline Backups (to S3)
The team set up replication to another AWS Region using S3 Cross-Region Replication. This was a Snowflake-native solution, implemented in Flyway and Terraform.
phData also set up job audit and error reporting via PAMS, our proprietary tool (offered for free in client projects).
Understanding performance, errors, and costs of data products can be tedious and difficult. PAMS is a set of packaged tools to aggregate data from data products to solve this problem. Learn more about PAMS here.Â
Customer Events Disaster Recovery
phData established the backup and recovery of both Kafka and Kinesis events to AWS S3. The team implemented the process using Lambda and Terraform, setting up job audit and error reporting via PAMS.
Apply Configuration from 3-Tier Assessment to Snowflake
With the tiered pricing from the previous engagement, the customer needed a way to reflect these changes in the backup and recovery process. This acts as a cost-savings measure and makes the workflows more configurable moving forward.
phData set up warehousing sharing and sizing rules for the backup and recovery process using Flyway.
phData set up a sustainable backup and recovery process for the customer, coming in ahead of schedule and under budget.
With this new process in place, the customer has what they need to meet compliance requirements within the healthcare IT space.
phData also provided documentation and training for the backup and restoration process. Walking the customer through what the recovery process will look like means they’re set up for maintaining the right steps moving forward.Â
Looking to automate your backup and disaster recovery process? Learn how phData can help!
Subscribe to our newsletter
Data Coach is our premium analytics training program with one-on-one coaching from renowned experts.
