Trigger Tests on Testsigma and Deploy on passed tests after CodeShip Builds

Testsigma offers powerful Web hooks for triggering Testsigma Executions remotely from any third-party tool that provides a Command Line Interface(CLI). In this guide, let us see how we can integrate Testsigma in Codeship for automatically triggering Test Executions with successful builds on CodeShip and wait for the tests to pass and then proceed with deployment.



1. A CodeShip Account containing your Organisation and the Project that needs to be triggered.

If you are new to CodeShip, please follow the following link to create an Account(using your email or OAuth), Add your Organisation and Create a Project - Signing up for new Codeship Account

2. Connect your Source Code Management(SCM) tool such as Github, BitBucket or GitLab to your account.

In this Guide, we will be using GitHub as our SCM. Therefore, we need to install CodeShip App from GitHub Marketplace in our GitHub Account so that the repositories in the Github account are accessible.

Please check the below guide for configuring Github as your SCM in CodeShip - Build Faster with GitHub and CloudBees CodeShip

For help on configuring other SCMs, please check CodeShip Documentation or contact the CodeShip Support.

You can view the Connected Services at Authentications page - Connected Services


Integrating Testsigma to CodeShip Test Pipelines

1. Click on Projects link on the CodeShip Home page and then select your required Project. The one we are selecting is 'testsigma-demo' in this case.

2. From the Dashboard page for your selected Project, click on Project Settings.

Clicking on Project Settings takes us to the 'Configure your Tests' page.

3. Select the option 'I want to create my own custom commands' in the Select your technology selector box.

4. In Setup Commands, enter the below command  to install jq package:

sudo apt-get install jq

Info: jq is json query parser library for bash

5. Later, click on Add Pipeline and enter a name for the pipeline. We will be entering 'Testsigma Execution Trigger'.

After that, refresh the page and then the 'test commands' window appears as shown above.

6. Go to Testsigma Executions page to get the Execution ID and CI Webhook URL as shown below:

7. Replace the <execution-ID>, <Username> and <Password> in the following script:


echo "Triggering Execution!"

# Triggering an execution using the <Execution-ID> from Testsigma Execution details page. Use your account login details as <Username> and <Password>
currentrunid=$(curl -sS -X POST -H 'Content-type: application/json' -u <Username:<Password><Execution-ID>/run)

echo "Started Execution. Waiting for tests to complete..."


#Use the Execution Status Check API and loop until the value of 'status' in the API Response equals to 0.
while [ $currentstatus -ne 0 ]; do currentstatus=$(curl -sS -X GET -H 'Content-type: application/json' -u <Username:<Password><Execution-ID>/run/$currentrunid/status | jq '.status'); sleep 5; done

echo "Test Execution Completed. Checking results..."

#Get the value of 'result' from the API Response for Execution Status Check API
currentresult=$(curl -sS -X GET -H 'Content-type: application/json' -u <Username:<Password><Execution-ID>/run/$currentrunid/status | jq '.result')

echo $currentresult

#Check the result of the execution using Execution Status Check API - If the value of 'result' is 0, the test passed. Else, the test failed.
if [ $currentresult -ne 0 ]; then echo "Test Failed. Please check Testsigma to know the reason for failure."; else echo "Test Passed! You may deploy now!"; fi

#Return the result as exit code - Non-zero value for execution result depicts Test failure and also triggers this script failure.
$(exit $currentresult)

8. Paste the script in the new 'test commands' box as shown below and click on Save and go to Dashboard button.

9. Now go to the Project Settings > Build Trigger to add a build Trigger.

10. Optional - You may also go to Project Settings > Deploy to add details of your deployment platform.

That's all we need to do for triggering the Testsigma Executions in the CodeShip Pipeline. With the above settings, the build will be triggered whenever there is a change in the repository and as soon as the build is triggered, the tests will begin in Testsigma. The script will wait until the test completes and checks that the result of the tests is passed.

Happy Test Automation!