Take a moment to imagine a scenario where a Snowflake Admin team does not exist at all. Each internal team interested in using the Snowflake Data Cloud would have to start their journey from scratch. There would be no one holding the guardrails, no one to emanate a beacon from where guiding principles can be gleaned from, and there would quickly be as many different patterns of using Snowflake as there are teams using it.
What Does the Snowflake Admin Team Look Like?
In the world of Snowflake Administration, nothing akin to bad guys actually exists. That’s not to say that there aren’t times when we’re struggling through a problem and it can feel like we’re trying to prevent Thanos from snapping his fingers with all five infinity stones in his gauntlet. There most certainly are, however, heroes. Heroes that each have their own core strengths and very much like the Avengers, can only solve so many problems single-handedly.Â
Cheesy Avengers metaphors aside, the point we’re trying to make is that building a Snowflake admin team is like assembling anything from a puzzle, to a recipe, to a computer. Each piece, each member, plays a critical role and the whole is incomplete if left out.
In our experience helping organizations of all sizes build better data products with Snowflake, there are the four key roles our most prosperous customers leverage to build a successful Snowflake administration team:
- Solutions Architect (Captain America)
- DevOps Engineer (Iron Man)
- Ops team (err… all the other Avengers?)Â
- PM (Agent Coulson)
In this blog, we’ll explore how each role supports the overall mission of optimizing success with the Snowflake platform — all while using relatable Avengers analogies.
Solutions Architect
The Solutions Architect is without a doubt the Captain America of the team. This role is a mentor, the team leader, and the foundation for everything that follows. An incomplete fulfillment at this level will lead to cascading problems down the line. If the wrong approach, tools, or design decision is moved forward, it won’t matter how good the DevOps Engineer or Ops team is.Â
The Solution Architect is the person with the deepest knowledge and experience with Snowflake. They will help to avoid costly pitfalls by having a thorough understanding of the business use cases, info architecture, and how to mold that successfully into Snowflake’s methodology.Â
As workloads move to Snowflake, inevitably, there will be forks in the road and decisions to be made. The Solution Architect will rely on their experience for steering the ship down the right path.Â
For example, when automated Snowpipe ingestion from S3, should SNS or SQS be used? For everyday user and application administration, should automation be deployed (hint: the answer is always yes)? What is the best approach and criteria for monitoring and scaling warehouses? Do your projects have complementary cloud services that require building and deployment around Snowflake?Â
This role works in absolute lockstep with the next role we’ll discuss, the DevOps Engineer.
DevOps Engineer (Iron Man)
The Tony Stark of the team (minus the private jet) is without a doubt the DevOps Engineer. Like Mr. Stark, the DevOps Engineer has a mindset for automating the right things, the right way. They will gladly spend two hours developing a solution to do a task they could complete manually in 15 minutes. Maybe not to the level of building Jarvis (but one can dream). These folks get that there are true time savings and reliability gains to be had down the road because rarely are tasks one time asks.
Similar to the Solutions Architect, this person also has a deep working knowledge of Snowflake. On top of that are layers of GitOps, programming, SQL, and cloud Infrastructure as Code (IaC) skills.
Wait a minute, why a GitOps skillset? Isn’t Snowflake a saas offering?Â
Absolutely, and a terrific one at that. However, there still remains a necessity to process deployments/changes to databases, storage integrations, resource monitors, roles, grants, warehouses, etc. while maintaining standards.
We sometimes work with clients where it’s not unusual to receive a single request for hundreds of new roles. With the subsequent number of grants and objects to provision, this type of request can warrant thousands of lines of SQL. That’s not even considering the potential need to provision new AD groups. Using a tool like Tram with proper CI/CD pipelines and safety checks will help keep things manageable (and Ops Team members sane).
How can cloud infrastructure (S3 roles/policies, Azure Event Grid subscriptions, etc.) be deployed in a time-efficient, repeatable manner by the Ops Team with minimal risk of errors?Â
This is another question the DevOps Engineer would help implement the solution for. Whether it would be deploying CloudFormation for AWS stack creation or Terraform for Azure resource provisioning, this role will get the necessary groundwork in place. phData has also built Cloud Foundation to help integrate the best practices and templates required to support CI/CD on these tasks.
One last note on the DevOps Engineer; in the real world, there is much more to the role than just implementing solutions or updating YAML files all day. They can often be called upon to wear the iron helmet (hat) of an escalation point for the Ops Team. While they may be neck-deep in YAML, the DevOps Engineer also brings to the table a skill to quickly/comfortably switch gears and jump in to help troubleshoot, for example, an OAuth Tableau to Snowflake integration problem.
Ops Team
At the end of the day, the DevOps engineer and Solution Architect can only hold down the fort (Avengers HQ) for so long as Snowflake users and use cases begin to scale. Eventually, more boots on the ground are warranted and it’s time to call in the cavalry. Time for Dr. Strange and the mystics to portal in Falcon, the Wakanda Warriors, and the Guardians of the Galaxy. In the world of Snowflake Administration, this is particularly crucial if a need arises for support in multiple geo-locations with 24/7 support.
These team members have a developing knowledge of Snowflake. These are the folks that will, for example, respond to a notification off-hours for a Resource Monitor reaching its credit quota. They will be familiar with hunting down a rogue cartesian product query, a long-running oversized warehouse, or identifying a need to simply increase a credit quota to keep things running smoothly or potentially saving unnecessary credit burn.
The Ops Team will also be responsible for living in the IT service management (ITSM) system, processing requests/incidents, and meeting SLAs. To accomplish this, possessing a thorough working knowledge for both using and troubleshooting all available tools is key.
Remember those complimentary cloud tools that the DevOps Engineer built in the last section? These are also important to the Snowflake platform success because they are driving everything from pipeline ingestion to orchestration and governance. Adding on new technologies increases the complexity to troubleshoot and the Ops Team is here to support the Snowflake ecosystem.
PM or Delivery Manager
Anyone familiar with Avengers is also familiar with the beloved Agent Coulson. He has a knack for being the heart of a team. For ensuring the individual members keep a larger perspective on working together. While a PM may not be the strongest Avenger technically, keeping the team communicating, on task, and on schedule is invaluable. You can have the strongest of superheroes on the team but if they’re all working independently, things can quickly turn to chaos.
What if I Don't Have All These Roles at my Organization?
It’s important to emphasize these roles cannot be cherry-picked if the goal is to put together a Snowflake Admin team that is whole.Â
Imagine a scenario without a DevOps Engineer. The Ops Team would quickly be overwhelmed with repetitive tasks. Misconfigurations would inevitably be a way of life, and valuable time would soon be diverted to fixing them. Now imagine a situation without an Ops Team. Despite automated solutions in place, the Solutions Architect and DevOps Engineer would quickly be overwhelmed. Time would be spent on the day-to-day instead of investing time into potential future enhancements.
Upskill your Employees
It’s certainly possible to learn the necessary skills. Just like learning anything, it will take a dedication of time, patience, budget, and practice. Providing a full syllabus of training items is beyond the scope of this post, however, here are a few helpful links to start the journey:
- Snowflake University (On Demand)
- Snowflake guided training course
- Tram Admin Guide
- Getting Started with IaC (Terraform): AWS, Azure, GCP
Lean on a Team of Experts to Execute on Your Behalf
Our phData Snowflake Administration service provides reliable security, monitoring, and support for your Snowflake platform. phData is essentially like adding the Avengers varsity squad to your Snowflake account. Our experience and specialized skills can help your business extract the most value from your Snowflake account.
For example, we provide:
- Continuous monitoring and support to deliver a reliable platform for developers, analysts, and data scientists.
- Expert implementation and configuration support on new, complex data and machine learning technologies used with Snowflake.
- Cost savings on your platform administration costs, on average, around 45%.
Discover how combining forces with phData can noticeably help with Snowflake Administration.Â
