Synapse Minimal Test Framework – Part 4: CI with GitHub

In this 5 part series I will be talking about the Synapse Minimal Test Framework. The why, how and whatnot:

• Part 1: Let’s get testing – Introduction, considerations, and approach
• Part 2: Simple pipeline test – Testing a simple unzip pipeline using Storage
• Part 3: Advanced pipeline test – Testing a complex pipeline using SQL
• Part 4: CI with Github – Automate testing using Github Actions
• Part 5: CI with Azure DevOps – Automate testing using Azure DevOps Pipelines

Want to dive straight into the code?

Want to skip reading? All code below can be found on GitHub: jpaarhuis/synapse-minimal-test-framework: Minimal test framework to test Azure Synapse pipelines with MSTest (CI included) (github.com)

Automate testing using Github Actions

In the previous parts we wrote tests to test our Synapse Pipelines from our own MSTest project. In today’s post we will talk about how to run these tests automatically on repository updates or on specific scheduled times.

GitHub Actions is a powerful and flexible automation platform that can be used to build, test, and deploy software applications. In this blog post, we will explain the workflow for running tests on GitHub and generating a test report.

GitHub Workflow for Running Tests

The GitHub workflow is a YAML file that describes a set of actions to be taken when certain events occur in a repository. In this case, we want to run our tests when code is pushed to the repository’s main branch.

Here is the YAML code for our GitHub workflow:

name: Run Tests

on:
  workflow_dispatch:
  push:
    branches:
      - main

permissions:
  id-token: write
  contents: read
  checks: write

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2

    - uses: azure/login@v1
      with:
        client-id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant-id: ${{ secrets.AZURE_TENANT_ID }}
        subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} 
        enable-AzPSSession: true

    - name: Setup .NET 6.x
      uses: actions/setup-dotnet@v3
      with:
        dotnet-version: '6.x'

    - name: Run Tests
      run: dotnet test --logger "trx;LogFileName=test-results.trx" || true

    - uses: dorny/test-reporter@v1
      with:
        name: Pipeline Tests
        path: "**/test-results.trx" 
        reporter: dotnet-trx
        fail-on-error: true

Let’s go through this code section by section.

Workflow Trigger

The “on” section specifies the events that will trigger our workflow. In this case, we want the workflow to be triggered when code is pushed to the repository’s main branch. We also added a workflow_dispatch event that allows us to manually trigger the workflow from the GitHub Actions UI. You could also use scheduled triggers here.

Permissions

Next, we specify the permissions required for our workflow. We need write access to the repository’s status checks, read access to its contents, and write access to its checks. This will allow our workflow to update the status of the repository’s checks using the test results.

Steps

The “steps” section contains the set of actions that will be taken in our job. In this case, we have the following steps:

Checkout

The first step checks out the repository code using the “actions/checkout@v2” action.

- uses: actions/checkout@v2

Azure Login

The second step uses the “azure/login@v1” action to log in to the Azure account using the secrets stored in our repository.

- uses: azure/login@v1
  with:
    client-id: ${{ secrets.AZURE_CLIENT_ID }}
    tenant-id: ${{ secrets.AZURE_TENANT_ID }}
    subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    enable-AzPSSession: true

This step requires the “AZURE_CLIENT_ID”, “AZURE_TENANT_ID”, and “AZURE_SUBSCRIPTION_ID” secrets to be set up in our repository settings.

To get those properties you will have to create an Azure Active Directory application and service principal. Create an Azure AD app and service principal in the portal – Microsoft Entra | Microsoft Learn

.NET Setup

The third step sets up .NET 6.x by using the “actions/setup-dotnet@v3” action.

- name: Setup .NET 6.x
  uses: actions/setup-dotnet@v3
  with:
    dotnet-version: '6.x'

Run Tests

The fourth step runs the tests using the “dotnet test” command and saves the test results to a file named “test-results.trx”. We added “|| true” at the end of the command to ensure that the workflow doesn’t fail if the tests fail.

- name: Run Tests
  run: dotnet test --logger "trx;LogFileName=test-results.trx" || true

Test Reporting

Finally, we use the “dorny/test-reporter@v1” action to report the test results. This step uses the “test-results.trx” file that is created in the previous step.

- uses: dorny/test-reporter@v1
  with:
    name: Pipeline Tests
    path: "**/test-results.trx"
    reporter: dotnet-trx
    fail-on-error: true

Result

When everything is working correctly, and all tests succeed you will see a test report under the “Pipeline Tests” job of the workflow with a lot of green check marks.

However, if one or more tests fail it will be more red and show you which tests are defect and explain what it would have expected.

Conclusion

In this blog post, we discussed how to set up a GitHub workflow to run tests automatically when code is pushed to the repository’s main branch. We also showed how to use the “dorny/test-reporter” action to generate a test report based on the results of our tests. By automating the testing process, we can ensure that our code is always tested and ready for deployment.

In the next and final post we will discuss how we can do the same, but then with CI in Azure DevOps.