How do you write software that tests itself? We at Testup are asking us this question every day, because we are writing software that allows everybody to test their software. And that obviously includes ourselves. One of our main missions is to write software that we love to use for testing our own software. Sounds twisted? Well, ….
Let’s first look at the attributes that we love and who is the hero in that discipline.
| Attribute | Hero technology | Ideal |
| Fast response | Static code analysis | Get feedback as you type |
| Full history | Stack trace (Log file) | See everything that happened in run up to the failure |
| Transparent state | Debugger | See all variable values at the time of failure |
| Production like environment | Docker, Infrastructure as code | Quick reproduction of deployment settings |
| Smart assessments | Human AI | Make informed decisions about the validity of the observed behavior |
How can we achieve all these qualities at once? Unfortunately there is no hammer that hits all nails, at least not all at once. Let’s look at our build pipeline and see how we are doing.
There are three different environments to run the tests:
- Build environment
- Triggered with every commit (or locally on request)
- Has access to only the artefacts of one repository at a time
- Should finish within few minutes at most
- Integration environment
- Replicates all components of the productive environment
- Mocks/disables external components like payment, messaging, …
- Automated tests finish within an hour or less
- Staging environment:
- Usable environment for humans to interact
- Final validity check for new features
- Should be completed within a day
Below you can see the the build process from first commit up to production release. There are only two manual steps:
- Pull request: Manual inspection of code quality
- Acceptance test: Manual inspection of the feature’s usability before release
We will cover all aspects of the build pipeline in future posts. Next to come: “Automated UI tests”
“Testing with a Twist” is a series of articles about testing with Testup. Up to now the following articles were published:
