This post is a continuation of the series that examines the simplification of data integration leveraging Fivetran and Snowflake. Fivetran nearly eliminates the development time required to ingest data from a vast number of the most common data sources. Snowflake distinguishes itself by separating data storage from data processing, making both cost and performance optimal.
This post focuses on using the Jira connector to move Agile project management information into Snowflake in a structure conducive for creating reports off of the project data.
The real game-changer with Fivetran (and most major data replication tools), is the bonus of having full change data capture (CDC) capabilities, which accounts for inevitable source schema changes.
For the steps to create a Destination in Fivetran as well as setting up Snowflake of the connection, please reference this blog.
Step 1: Adding the Jira connector
The first step is logging into your Fivetran account then clicking on the “+ Connector” button to add a new connector. Search for Jira and select the option. Right now, there is only one type of connector, but that could potentially change as both softwares continue to grow.
After you select Jira, the page below will come up. The setup guide walks through what needs to be done to successfully set up the connector.
In this step, it’s important to populate the fields for Host and Port, then obtain the Public Key to link with your Jira account.
Step 2: Get Jira ready to export the data
After logging into your Jira account, go to the settings icon then select “Products.”
Then, click on “Application Links.”
After you select application links, then type in https://fivetran.com
Most likely you will receive an error message that no response was received. That’s ok and to be expected. The following screen will popup. If the Fivetran URL isn’t already populated, enter in the URL again and select “Continue.”
The following screen will come next. Type in “Fivetran” as the Application Name, Service Provider Name, Consumer key, and Shared Secret. Make sure “Generic Application” is selected for Application Type. Lastly, enter in the Fivetran URL for the Request Token URL, Access token URL, and Authorize URL fields. Don’t forget to check the “Create incoming link” box as well.
After completing that page, you will need to fill out the Consumer Key with “OauthKey” and the Consumer Name with “Fivetran”. For the Public Key, go back to your Fivetran set up page and copy / paste the Public Key that was generated after populating your Host and Port information.
Ta-da! Jira is ready to push data to Fivetran!
Step 3: Authorize the API
The next step is to go back to your Fivetran account and authorize the API. Simply click the “authorize API” button. When Fivetran can find a connection, you will get a pop up message similar to the one below. Click “Allow” to let Fivetran pull data from Jira.
When the connection is successful, you will see a message like this.
When you navigate back to the connector set up, you will see the following message. When all that is done, click “Save and Test.”
Step 4: Start the Sync with Fivetran
Now that the API is set up, the next thing to do is to start the initial sync. During this step, Fivetran will pull the data from Jira and put it into a database in Snowflake. To do so, simply click the “Start Initial Sync” button.
As the data is syncing, you will see the following icons pop up showing that the sync is in progress.
When the sync is complete, you will see the icons and text change to the following.
Step 5: View the Data in Snowflake
When you navigate to your Snowflake instance, you should now see all of the tables that are available from Jira.
Below is a preview of the issue types table.
Going through this exercise with Fivetran, Snowflake, and Jira was my first time leveraging these tools together. I am still amazed at how simple the entire process it. Extracting data from sources traditionally is a very laborious and technical process. Using the Fivetran connector is like magic with how user-friendly and fast it is to bring data into Snowflake from various sources.
I hope you enjoyed reading this blog and that it made Fivetran connectors a little easier for you. If you have any questions, please reach out to our team of experts!
