If your organization is utilizing Power BI Premium Capacity for your Power BI reporting, deployment pipelines are an easy feature available to help manage and move Power BI content through development, test, and production workspaces. Although the out-of-the-box deployment pipelines can be very useful, there are times when the deployment requirements necessitate using a customized deployment pipeline with additional capabilities.
In this blog, we will cover how to build an automated and customized CI/CD (Continuous Integration / Continuous Delivery) pipeline that integrates with Power BI using Azure DevOps.
Power BI Deployment Pipelines
The default Deployment Pipelines feature with Power BI Premium can be great for a lot of things; moving content through the development lifecycle, changing data sources, managing workspace content, and more.Â
Although they are great for straightforward deployments, there are several key elements that Power BI Deployment Pipelines do not allow for that can be done using a CI/CD pipeline:
- Version controlling is still a missing element for Power BI. At the time of writing this blog, there is no way to maintain versions of Power BI content within the platform.
- Release approvals are also lacking in Power BI. Besides providing deployment pipeline access only to a specific group of people, there isn’t a native approval process for deployment.
What are CI/CD Pipelines?
CI/CD stands for Continuous Integration and Continuous Delivery and is a practice designed to help development teams ensure reliable delivery of frequent changes to a product. In the case of Power BI, the products are datasets, reports, and dashboards.Â
When integrating a CI/CD pipeline with Power BI, the process is facilitated by using Azure DevOps. Integrating these platforms helps fill in the gaps that Power BI deployment pipelines isn’t capable of filling:
Automated version controlling by leveraging git repositories
This allows developers to make changes to the .PBIX files locally using a copy and then merging them with the production version.Deployment approvals for new releases.
Requiring a deployment approval by an admin or lead developer (or any other designated group or individual) provides a system of checks and balances for pushing content into production and therefore leads to quality Power BI content and trust in your reporting solutions.
Integrating Power BI with Azure DevOps
To automate and deploy Power BI content using a CI/CD Pipeline in Azure DevOps, follow the high-level steps below.
1. Create a Service Principal in Azure Active Directory (AAD)
A Service Principal is needed to authenticate Azure DevOps on Power BI Service.Â
- To create a Service Principal go to Azure Portal and navigate to the Azure Active Directory services page
- Go to App Registrations (left panel in AAD) and click on New Registrations
- Enter Application name and under Supported Account Types choose 1st option if DevOps project is in the same tenant otherwise select a different option. Click Register
- After registering the App: Open the App → navigate to the Overview section and save the Application (client) ID and Directory (tenant) ID somewhere safe.
2. Create a Secret Value For The Service Principal
- In the App, click on Certificates & Secrets on the left and then + New Client Secret
- Enter the description and expiration time for the Secret Value
Â
After saving the client secret, the value of the client will be displayed → copy this value and store it somewhere because YOU WILL NOT BE ABLE TO RETRIEVE THE KEY LATER! The key will be used in later steps.
3. Create a Security Group in Azure Active Directory (AAD)
The Service Principal does not have access to any of the Power BI Content or APIs. To provide this access, create a Security Group in AAD and add the Service Principal to that Security Group as a member of that group.
4. Allow Service Principals to Use Power BI APIs In The Power BI Service Admin Portal:
- Open the Tenant Settings and then navigate down to Developer Settings
- Enable Allow Service Principals to Use Power BI APIs
- Specify the Security Group created in the previous step and then click Apply
5. Grant Admin Access For Service Principals
Give the Service Principal Admin privileges in the Power BI Deployment Pipelines and the associated Development, Test, and Production workspaces.
6. Install Power BI Extensions From the Azure DevOps Marketplace
- Create a DevOps Project in Azure DevOps.
- Download and Install Power BI Automation Tools and Power BI Actions extensions from the Azure DevOps Marketplace.
7. Create an Azure GIT Repo & Commit .PBIX Files to the Repo
In the Azure DevOps Project → click on the drop-down and click on + New Repository
- Repository Type: Git
- Provide the Repository Name and click on Create
- Click on the vertical ellipsis and then Upload File(s)
- Browse for the .PBIX on your local machine, enter a comment, and then click on Commit → the file will be uploaded to the main branch
8. Configure the Build Pipeline in Azure DevOps
- Once the .PBIX file is uploaded to the main branch, the build pipeline runs and integrates the latest version of the .PBIX file into the Artifact folder. The artifact folder acts as a source for the subsequent Release Pipeline.
To configure the build pipeline:
- Click the blue Set Up Build button.
- Click on Starter Pipeline under Configure
- A YAML file will automatically be createdÂ
- Replace the script in the red box in the 1st screenshot below with the script in the green box in the 2nd screenshot – this script will publish the .PBIX file into the Artifact folder.
- Click Save and Run on the top right.
- Provide a Commit Message and click Save and Run → now the build pipeline will run and integrate the .PBIX file into the artifact folder.
9. Configure the Release Pipeline in Azure DevOps
To configure the Release Pipeline:
- Navigate to the Pipelines tab and then click on Releases and then New Pipeline
- Rename the pipeline in the next window.
Add the Artifact folder – this will contain the .PBIX files and act as the source for the release pipeline
In the Artifacts section:
- Click on + Add an ArtifactÂ
- Select the Project and the Source (build pipeline) from the dropdowns.
- Click the Add button
- In the Stages section, Click + Add a Stage. Click Empty Jobs
- Rename the Stage Name
After renaming the Stage:
- Click on the blue link for (1 job, 0 task)
- Click on + in the Agent Jobs section and search for  Power BI Actions and then click Add
A new Power BI Service connection needs to be created now:
- Click the + New button
- In the Edit Service Connection window, specify the Tenant ID, Client ID, and Client Secret (from registering the Service Principal App in Azure), and the Service Connection NameÂ
- Click Save
After creating the Service Connection:
- Click the Service Connection from the drop down.
- Select Action Item and Upload Power BI Report
- Provide the name of the Development Workspace under Workspace Name
- Select the path of the source file under Source File
- Click the Save button on the top
Create another stage to deploy the Power BI content to the Test/UAT workspace:
- Click the + Add button and rename the stage
- Click on the blue link for (1 job, 0 task)
- Click + in the Agent Jobs and search for Deploy content in deployment pipeline  and click on Add
Create another Power BI Service Connection
- In the Edit Service Connection window, specify the Service Principal ID (Client ID), Service Principal Key (Client Secret), and the Tenant ID from registering the Service Principal App in Azure)
- Enter the Service Connection Name and click the Save button
After creating the new Power BI Service Connection:
- Click the Service Connection from the drop-down.
- Provide the name of the pipeline under Pipeline and select Target Stage as Test from the drop-down
- Under Artifacts to deploy select All if all the content should be deployed from Dev to Test/UAT – if only selective content should be deployed, select Selective
- Provide the name of the content to deploy to Test/UAT
- Click Save on the top
- Similarly, create the Third Stage to Deploy Content from Test/UAT to Production. Make sure to select the Target Stage as Production while. Leave all other fields as is and save the stage.
Deploying with Azure DevOps
Navigate back to the Pipeline and click on Create Release on the top
- If you want to change the trigger for any stage from automated to manual, select those stages under stages for trigger change from automated to manualÂ
- Select the latest version of the source file.
- Give the Release DescriptionÂ
- Click Create
- A new release will be created – click on the Release-1Â link to view the release
- Now the Test.pbix file is deployed into the Development workspace
When the content is ready for testing:
- Click the Deploy button on the Deploy to Test stage to deploy the content from the Development workspace to the Test/UAT workspace
- Repeat the previous step in Deploy to Prod stage when UAT is accepted to deploy content from the Test/UAT workspace to the Production workspace.
The content has now been deployed to all three workspaces.
Important Notes:
- There is a 5 GB limit on push size for very large pushes. To know more about GIT limits, refer to Git limits – Azure Repos – Microsoft Learn
- The only exception of a GIT push over the 5GB accepted is the import of a repository through the web interfaceÂ
- If you will constantly work with pushes over 5 GB (or very often) It is recommended to utilize GIT-LFS.
- To manage large files, refer to Microsoft
Conclusion
Using CI/CD pipelines in Azure DevOps with Power BI, you can automate version controlling of .PBIX files and automate the deployment process with additional customization. CI/CD pipelines can also be used to perform certain activities like creating a Power BI workspace, adding a new Admin user to a Power BI workspace, refreshing a dataset, managing Power BI pipelines from start to finish, assigning users to a pipeline, assigning users to a workspace, and more.Â
Looking for best practices or additional guidance on customizing or automating Power BI Deployments with CI/CD Pipelines?
