The Snowflake Data Cloud has vastly simplified how our customers interact with their data.Â
There are several features that simplify adoptions and ease of use so that once our customers get past the initial getting started phase in Snowflake, their focus more often than not shifts to the topic of budget and cost.
One recurring customer question from our Snowflake customers is about charging back credit consumption costs to the business unit that is utilizing the work.Â
While this is not a unique concept to Snowflake, our experience shows there is a unique process required to make this an effective approach on Snowflake.Â
In a phrase? It’s about building dashboards for Snowflake credits that track trending charges for an expanding list of operational activities within the platform. Only with this deep level of insight can you enable your chargeback model and democratize the knowledge of where costs are being accrued.
What is a Chargeback in Snowflake?
A chargeback in Snowflake is a model that tracks your costs and credit consumption back to the business unit that incurred it — based on actual usage.
This prevents your central IT team from taking on the entire budgeting burden of Snowflake and instead, ensures that your teams are paying for what they use.
With granular dashboards and reports in place, you’ll be able to budget more effectively by defining a chargeback model so you can monitor and optimize your credit consumption not only by warehouse but also by business unit or division.
To help solve these business challenges, we recommend creating two different types of dashboards: chargeback dashboards and operational dashboards.
The chargeback dashboards allow you to track credits to specific business units and provide them with hard data on their usage.
The operational dashboards provide additional detail into how those credits are being consumed. This information will help you track adoption and usage trends, optimizing Snowflake performance accordingly.
Let’s walk through what it looks like to set these two dashboards up to support a Snowflake chargeback model.
How Do I Set Up Snowflake and Dashboards to Track Credit Usage?
To make these dashboards functional, we created Snowflake views over the standard Snowflake tables that come with every account — namely, the ACCOUNT_USAGE and INFORMATION SCHEMA tables.Â
Because credit consumption in Snowflake is tracked at the warehouse level, we can create customized, hierarchical views for each warehouse by company, division, or business unit. Â
We’ll then apply row-level security and grants to those views so your end-users can report on specific activity based on their permissions within the organization.
phData has created standard dashboards over those views that you can customize to your specific needs.
What Can Snowflake Chargeback Dashboards Look Like?
To give you an idea of the kinds of insights you can pull together in both your chargeback and operational dashboards, we created some example dashboards that have helped other Snowflake customers successfully build out their chargeback model.
Overall Credit Consumption Dashboard
Creating an Overall Credit Consumption Dashboard allows you to track usage over time.Â
You can easily see which warehouses are consuming the most credits and the trajectory of credit consumption by warehouse or business unit. You can drill into unit-specific warehouses or view overall activity for your company.
The first dashboard shows overall credit consumption for the account. It includes the following information:
- Current Credit Consumption for the month
- Trending Month-over-Month consumption
- Credit Consumption by Time of Day
- Current Resource Monitors usage vs Credit Quota
The Filters or Slicers available for drilling down with this dashboard are:
- Environment (Prod, QA, Dev)
- Business Unit
- Warehouse
- Date
Credit Detail by Warehouse
The Credit Detail by Warehouse dashboard is most useful when you need to report back to business units or divisions on their specific activity and trending activity in Snowflake.Â
This helps create an open dialogue with the business on how user and job activity is trending and it helps your Snowflake platform owners prepare for future usage.
If you’re looking for more specific information than overall company usage, you can use these dashboards to drill down and view specific credit details by warehouse. Â
By default, this view shows credit consumption by month, time of day, and day of the week. This view can be used to track trends over time or zoom into specific time windows to see particular activity details.
Query Execution by Warehouse
The Query Execution by Warehouse dashboard will show you trending data and give you a more detailed analysis of the activity in your Snowflake account.Â
This can be used to uncover specific issues in your environment, such as long-running Snowflake queries (a major budget buster). The information can be used to optimize your platform and continue to drive more adoption.
These are some of the insights you can glean from this dashboard:
- Specific execution trends by warehouseÂ
- Specific characteristics of queries in a warehouseÂ
- Compilation times, to uncover complex queries that require higher-than-average compute timeÂ
- Execution times to uncover bulk queries that require larger-than-average data retrievals
- Â Ways to adjust your clustering or warehouse size to improve query performance
The example dashboard above indicates two very different usage patterns in December and January. In December, we see longer execution times for fewer queries while January shows lower execution times but a big increase in query counts.
In Snowflake, it’s important to try to group similar query patterns to the same warehouses so they can all use the settings that optimize their query times.Â
For example, large, complex queries may need to take advantage of Snowflake’s auto-clustering functionality. A large quantity of smaller executions may perform best in a larger size warehouse to take advantage of Snowflake’s automatic parallel execution.
How Else Can I Use These Dashboards?
These three dashboards are just the beginning of how you can start utilizing your Snowflake data for optimization and cost savings.
We’ve only scratched the surface of what these example dashboards show. The dashboards above can be expanded to offer other operational analysis, such as:
- Analysis of specific queries to track compilation time, execution time, and queuing (Queuing trends may indicate some fine-tuning for a specific warehouse, while peak usage times may indicate an opportunity to spread out scheduled jobs)
- User analysis to show most active users and usage patterns by user
- Inactive users over time to help track adoption among user groups
- Error analysis to track the kinds of errors users may be facing and identify training opportunities
- Object analysis to track activity on database objects such as databases, views, and tables
The chargeback and operational Snowflake dashboards are a proactive way to track usage trending in your Snowflake account. Knowing how users are using the platform allows our customers to adjust settings in the account for optimal performance.
Looking for more Snowflake best practices?
Snowflake Chargeback and Dashboards FAQs
The views and dashboards we’ve built are intended to kick-start Snowflake customers. We provide a template and can help deploy that template, but end-users typically modify the template reports and enhance them based on their specific needs. We are constantly updating and enhancing these templates and are always willing to share new ways of viewing the data.
No, these views are intended to provide greater activity trends but not real-time views. The underlying Snowflake tables that support these dashboards have some latency built into them based on how they are updated in Snowflake. That, combined with different ways to import the data into individual BI tools, is why this data is not necessarily updated in real-time.Â
Yes. Any BI tool can be used on top of the views we’ve created. We’ve standardized on Power BI, but any BI or SQL tool can be used on top of the Snowflake views.
