Author: Andreas Grau

For the last 14 years, I have been working on tools for developers at Thetaris. My Co-founder Stefan and I developed, we developed all kinds of tools, from Excel plugins, financial mathematics libraries, test suites, apps to a full-fledged IDE for our own programming language ThetaML. During this work, we found that many developers in our community struggle with maintaining their applications. The developers fix a bug or implement a new requirement and suddenly, a seemingly unrelated feature of the application breaks.

Unit Testing is part of the solution

There should be an easy answer: testing. From the large projects we completed at Thetaris, we know that Unit-Tests are part of the solution. They are quick and can be used to automatically stop bugs from entering the code case. But they are not sufficient. The applications still break.

Manual testing is mandatory

Manual testing is an expensive and time-consuming solution. However, it works. The testers identify not only functional issues, but also UX glitches and text typos. Consequently, testers need the focus of attention. The testers should be enabled to maintain their own automated tests. Over the years, we tried many different approaches to reach this goal.

Selenium-Python, Cucumber

I thought, why not make my testers to QA engineers. We created a Python library which uses a simple language such that test cases are easy to understand and easy to extend. In other projects we used Cucumber, a simple language to make test cases easy to understand. Both approaches delivered the same result: A QA engineer is required for the language definition and testers can run tests. In some cases, tester can add test themselves. But, usually, they need a developer again who changes some of the application code.

Visual Testing

Why would that be? Why can testers not just test the software without knowledge of the internal workings? I thought that a manual tester uses the User Interface, only. This must be completely sufficient to write automated tests using the UI. Using my computer vision experiences, I created a prototype in Python to provide a proof of concept for a visual test automation. It worked surprisingly well. No inner knowledge of the application was necessary anymore.

Testing in Cloud

Another main issue testers face when running automated tests is the constantly changing infrastructure: E.g. Google Chrome gets an update and the Selenium Driver breaks. The connection of the native app using Appium on the local desk is another constant source of issues. Working in the cloud with Browserstack for native devices and Mabl for websites shows that testing belongs in the cloud. Testers are more productive and happier when they do not have to care about the test system infrastructure.

Simplicity

The last missing piece to a wide acceptance of test tools by testers is simplicity. I am lucky that my Co-founder is a UX nerd. During the development of our own test platform, Stefan included iteration after iteration to minimize the required user interaction for test recording. Minimizing the decisions a user has to make, we are also minimizing errors the user can do. We constantly collect feedback from new users of our application and simplify it further.

Putting it all together: Testup

Now, we have put it all together in the Testup project: Visual Testing using Computer Vision, Infrastructure in the Cloud and Simplicity by No-Code test creation. The effort is fruitful: Test are easy to create, easy to ready and easy to change. The next step is the version 0.9 which we will release in a few days with an open beta. We are looking forward to your feedback!