Posted on

CI-CD Results API

This page lists the API calls that are meant for external integration. Currently there is one endpoint to get information in JSON format.

Also look at Integrate testup into the CICD pipeline, for API calls to start schedules and get response in plain

Authentication

All calls to our api can be authorized via the ApiKey. You can get this key from your user, or your project settings page. Please check for a description on to get this ApiKey in step one of cicd integration. For following we assume that $TESTUP_APIKEY is set to your key.

To access any REST endpoints you need to set the following header

Authentication: ApiKey-v1 $TESTUP_APIKEY

In curl commands you can set the Header like this:

curl … -H “Authentication: ApiKey-v1 $TESTUP_APIKEY”

Project status

The project status call returns the result of the previous test runs. You can call the following address to the get the current status

https://app.testup.io/cicd/project/$PROJECT_ID/results


The following query parameters are possible:

limitNumber of past executions that are included in the response (default 20)
fromDatestart time of the execution list (Iso format: YYYY-mm-DDTHH:MM:SS)
toDateend time of the execution list (Iso format: YYYY-mm-DDTHH:MM:SS)

Example:

curl “https://app.testup.io/cicd/project/$PROJECT_ID/results?limit=1” -H “Authorization: ApiKey-v1 $TESTUP_APIKEY”

Response:

{
  "project": {
    "id": 1,
    "name": "Project name"
  },
  "executions": [
    {
      "id": 2,
      "name": "Run by Schedule",
      "startTime": "2023-10-18T16:15:00.239571Z",
      "tests": [
        {
          "id": 3,
          "name": "Test case one",
          "status": "PASSED",
          "failedRecordings": []
        },
        {
          "id": 4,
          "name": "Test case two",
          "status": "PASSED",
          "failedRecordings": []
        }
      ]
    }
  ]
}
Posted on Leave a comment

QA Bite ::: Write a Clear and Simple Test Report

In large projects automated test reports are often not sufficient. Stake holders of the project are not familiar with the specific cases tested and the severity of the bugs found. Also, from a business perspective, it is often much better to release the software with known issues than waiting for a fix. Thus, the test report must enable the product owner to judge whether the new software version is better than the old one, what can and can’t be done with it and the implications for the User Experience.

Posted on Leave a comment

QA Bite ::: Unit test your Applications

Unit testing involves the individual testing of all the components of an application or website to ensure that the application meets its design requirements and behaves as intended. Since unit testing is the most intensive test that can be carried out on an application, it is often disregarded by software developers and considered time-consuming. But is Unit testing still worth the time and effort?

There are many latent benefits of performing Unit testing. It enables developers to make big changes to code quickly and also gives confidence that all units/components work exactly as they should. It also helps in finding problems early in the development cycle and helps with the documentation of coding progress, enabling developers to learn from previous mistakes.

Contrary to popular opinion, unit testing speeds up the development process as it makes it easy for developers to reuse/migrate code components more efficiently as the unit tests of these components can also be migrated as well.

Posted on Leave a comment

QA Bite ::: Test Every Possible Scenario and More

In many situations, it was very helpful that someone dedicated to testing performed seemingly ineffective tests with somewhat random test steps. This way, the application is tested for both, for what it is supposed to do, and for what it is not supposed to do. You can even go a little further and use Fuzzing, that means completely random input. This way, you can find many security vulnerabilities.

Posted on Leave a comment

QA Bite ::: Developers Should Not Write Tests

This is no offence to the developers. Developers are as close to the code as it gets. Thus, they should write unit tests. But, developers should not write End-to-End tests. This is to reduce the chances of the developer running a biased tests or a lack of creativity in testing. Someone who knows what the software will be used for must write the tests. This is usually someone from the product team, e.g. the Product Owner.

Posted on Leave a comment

QA Bite ::: Run Tests at Every Stage of the Development cycle

Early testing ensures quick detection of bugs and reduces time and energy required to perform subsequent tests. It also builds confidence in developers to include new features in the product.

Client Needs are covered by Acceptance Testing.

Requirements are handled by End-to-End Testing.

Software Design is ensured by Integration Testing.

Code is checked by Unit Testing.

Posted on Leave a comment

QA Bite ::: Use Combinatorial Testing to Reduce the Number of Test Cases

In practice, it is usually impossible to test all parameter combinations for real life applications. Say, you have 10 different parameters of a tax advice software where each parameter is true of false. Enumerating all parameter combinations leads to 3,628,800 test cases already. So, testing can always check a few combinations, only.

But, when the software finally goes public, I often see that rare usage patterns occur which have never been tested. There is a neat trick by a research group at NIST: Automated Combinatorial Testing for Software. Richard Kuhn, Raghu Kacker and Yu Lei observed that most errors occur by a change of one or two parameters, only. A two way test with 10 parameters can be done in 10 test cases. Even a 4-way test, where a specific set of 4 parameters is required to trigger a bug, is easy to do with 42 test cases.

Posted on Leave a comment

QA Bite ::: Integrate Testing into your Product Development Roadmap

product roadmap is a strategic document that maps out the general stages of a product’s development. In startups or new businesses, a product roadmap usually serves as a filter to determine the most important tasks involved in a product’s development.

In Agile product development, activities such as split A/B testing are usually omitted from the product roadmap. This is because Agile product development is time-bound, hence no extra time is dedicated to testing which is often regarded as time-consuming.

To integrate testing into a product roadmap, the product management team first needs to understand and accept that testing is one of the most important tasks involved in product development, hence it deserves to be included in the roadmap. Also, the Engineering team should be duly included in the creation of the product roadmap from the beginning. This will ensure that Testing is performed at every step of product development. Finally, to save time testing, product management teams are advised to consider Automated testing.