Trigger Tests on Testsigma and Deploy on passed tests after code change on Bitbucket/Github/any other supported SCM Tool

Testsigma offers powerful Web hooks for triggering Testsigma Executions remotely from any third-party tool that provides a Command Line Interface(CLI). In this guide, let us see how we can integrate Testsigma in CircleCI for automatically triggering Test Executions with code change on Bitbucket Repo and wait for the tests to pass and then proceed with deployment.



1. A CircleCI Account

2. Connect your Source Code Management(SCM) tool such as Github, BitBucket or GitLab to your account.

In this Guide, we will be using Bitbucket as our SCM.


Integrating Bitbucket/GitHub to CircleCI

During the on-boarding, you are asked to log into CircleCI using one of the SCM tool login - Github/Bitbucket.

Once you login with OAuth, the repositories in your SCM will be visible in CircleCI. You can set up any of them for Continuous Delivery using the 'Set Up Project' button.

1. Clicking on Set up Project takes us to the Set Up page as shown below:

2. Modify the config.yml file and include a job to run the bash script file that initiates Testsigma Execution and waits for its completion before deployment. Here's a sample entry for config.yml file:

# Java Maven CircleCI 2.0 configuration file
# Check for more details
version: 1.0
      # specify the version you desire here
      - image: circleci/openjdk:8-jdk
      - image: debian:stretch
      # Specify service dependencies here if necessary
      # CircleCI maintains a library of pre-built images
      # documented at
      # - image: circleci/postgres:9.4
    working_directory: ~/repo
      # Customize the JVM maximum heap limit
      MAVEN_OPTS: -Xmx3200m
      - checkout
      # Download and cache dependencies
      - restore_cache:
            - v1-dependencies-{{ checksum "pom.xml" }}
            # fallback to using the latest cache if no exact match is found
            - v1-dependencies-
      - run: mvn dependency:go-offline
      - save_cache:
            - ~/.m2
          key: v1-dependencies-{{ checksum "pom.xml" }}
      # run tests!
      # Trigger test on Testsigma and wait until completion or timeout!
      - run: chmod a+x ./
      - run: ./
      # deploy!
      - run: ./

3. Now, click on Start Building button on the top right to start a test build. You will be greeted with a dialog stating that the config.yml will be added to the selected Repo by CircleCI.

4. Click on Add Config button to Add the Config and start building. It will take you to the Pipelines page.

That's it. Integration of your Repo to CircleCI tool is complete. Commits to your repository would now trigger a build automatically.


Integrating Testsigma with CircleCI

The initial build of your Project will fail since we have not supplied the file mentioned in the config.yml file. We need to add that manually to the Repository. 

You can create an SH file from the following article: Generic Shell Script for Integrating with CI/CD Tools

Make the below modifications in the Script file:

1. To trigger a specific Test Plan as part of the CI/CD Pipeline, go to Test Plan details page to get the <Execution ID> and <Test Plan Trigger URL> as shown below:

2. Update your <Username>, <Password> and <Test Plan Trigger URL> in the above Generic Script before uploading that to the Repo.

Now add the file to the src folder of your Repository and commit.

With the above settings, the build will be triggered whenever there is a change in the repository and as soon as the build is triggered, the tests will begin in Testsigma. The script will wait until the test completes and checks that the result of the tests is passed.

Happy Test Automation!