Testing with a Twist: The Testup Build Pipeline

How do you write software that tests itself? We at Testup are asking us this question every day, because we are writing software that allows everybody to test their software. And that obviously includes ourselves. One of our main missions is to write software that we love to use for testing our own software. Sounds twisted? Well, ….

Let’s first look at the attributes that we love and who is the hero in that discipline.

How can we achieve all these qualities at once? Unfortunately there is no hammer that hits all nails, at least not all at once. Let’s look at our build pipeline and see how we are doing.

There are three different environments to run the tests:

• Build environment
  • Triggered with every commit (or locally on request)
  • Has access to only the artefacts of one repository at a time
  • Should finish within few minutes at most
• Integration environment
  • Replicates all components of the productive environment
  • Mocks/disables external components like payment, messaging, …
  • Automated tests finish within an hour or less
• Staging environment:
  • Usable environment for humans to interact
  • Final validity check for new features
  • Should be completed within a day

Below you can see the the build process from first commit up to production release. There are only two manual steps:

• Pull request: Manual inspection of code quality
• Acceptance test: Manual inspection of the feature’s usability before release

We will cover all aspects of the build pipeline in future posts. Next to come: “Automated UI tests”