Trigger Tests on Testsigma and Deploy on passed tests after CodeShip Builds

Testsigma offers powerful Web hooks for triggering Testsigma Executions remotely from any third-party tool that provides a Command Line Interface(CLI). In this guide, let us see how we can integrate Testsigma in Codeship for automatically triggering Test Executions with successful builds on CodeShip and wait for the tests to pass and then proceed with deployment.



1. A CodeShip Account containing your Organisation and the Project that needs to be triggered.

If you are new to CodeShip, please follow the following link to create an Account(using your email or OAuth), Add your Organisation and Create a Project - Signing up for new Codeship Account

2. Connect your Source Code Management(SCM) tool such as Github, BitBucket or GitLab to your account.

In this Guide, we will be using GitHub as our SCM. Therefore, we need to install CodeShip App from GitHub Marketplace in our GitHub Account so that the repositories in the Github account are accessible.

Please check the below guide for configuring Github as your SCM in CodeShip - Build Faster with GitHub and CloudBees CodeShip

For help on configuring other SCMs, please check CodeShip Documentation or contact the CodeShip Support.

You can view the Connected Services at Authentications page - Connected Services


Integrating Testsigma to CodeShip Test Pipelines

1. Click on Projects link on the CodeShip Home page and then select your required Project. The one we are selecting is 'testsigma-demo' in this case.

2. From the Dashboard page for your selected Project, click on Project Settings.

Clicking on Project Settings takes us to the 'Configure your Tests' page.

3. Select the option 'I want to create my own custom commands' in the Select your technology selector box.

4. In Setup Commands, enter the below command  to install jq package:

sudo apt-get install jq

Info: jq is json query parser library for bash

5. Later, click on Add Pipeline and enter a name for the pipeline. We will be entering 'Testsigma Execution Trigger'.

After that, refresh the page and then the 'test commands' window appears as shown above.

6. Obtain API Key and Test Plan ID from Testsigma App

a. If you have not created a Testsigma API key already, please create one by navigating to Settings > API Keys and clicking on Create an API Key button on the top right corner.

We will be using this API Key to authenticate our remote Test Plan trigger requests from CircleCI(explained in next article section - Set up Code Repository)

b. Create a Test Plan in Testsigma that you would like to execute as part of the CircleCI Pipeline. Obtain the Test Plan ID for the Test Plan from the CI/CD Integrations tab within Test Plan details as shown below:

We will be using this Test Plan ID to specify which Test Plan to trigger from CircleCI(explained in next article section - Set up Code Repository)

So, we have noted down the Testsigma API Key and the Test Plan ID now. We will be adding those to a Shell script and add the shell script file to your Code Repository in the next section.

7. Copy the shell script from the article: Generic Shell Script for Integrating with CI/CD Tools and customize below properties in the Shell script.

#********START USER_INPUTS ********

TESTSIGMA_API_KEY= <Give in an API Key Generated in the Configuration-> API Keys tab in Testsigma>

TESTSIGMA_TEST_PLAN_ID= <Test Plan ID for the run which you need to trigger>

#********END USER_INPUTS***********
Update your Testsigma API Key and Test Plan ID in the above script

8. Paste the script in the new 'test commands' box as shown below and click on Save and go to Dashboard button.

9. Now go to the Project Settings > Build Trigger to add a build Trigger.

10. Optional - You may also go to Project Settings > Deploy to add details of your deployment platform.

That's all we need to do for triggering the Testsigma Executions in the CodeShip Pipeline. With the above settings, the build will be triggered whenever there is a change in the repository and as soon as the build is triggered, the tests will begin in Testsigma. The script will wait until the test completes and checks that the result of the tests is passed.

Happy Test Automation!