Tag: testup

Posted on by — Leave a comment

Test Faster, Better and Automatically

Did you hear about automated testing? If no, you have come to the right place. In this article, we are going to discuss software testing, automated testing and a specific automated testing tool called “Testup”.

Let’s begin!

What is software testing

Software testing is the process of verifying whether the actual results of the software meet the expected ones. The actual result is the software as it is developed by developers, and the expected result is the requirements of the client. It does not only mean that the software is running properly but also that the software is displayed correctly. Further, software testing helps testers to identify errors, missing requirements, and unexpected results.

Without proper software testing, the software may be no longer valid and may cause lots of problems to the end-user, he might get annoyed and switch to a different application. Therefore, it is necessary to test the software thoroughly.

Software testing divides into two main groups named manual testing and automated testing. As the name suggested, manual testing executes test cases by a human tester without using any automation while automated testing executes test cases using automation tools.

Further, software testing methods can be grouped based on the “box” approach:

  • Black-box testing – that means that the tester does not have any knowledge of the internal implementation and also does not see the source code of the software. For black-box testing, testers do not need to have programming knowledge. As a result, clients are also involved in this type of testing.
  • White-box testing – that means that the tester has some knowledge of the internal implementation and also sees the source code of the software. For white-box testing, testers need to have programming knowledge.

Why automated testing

There are some limitations of manual testing compared to automated testing such as

  • a high amount of time to complete a test,
  • consumes a lot of human resources, as a tester always need to be present,
  • the scope of the test is limited,
  • no support for performance testing and
  • the possibility of less accurate test results as it may involve human error.

Therefore, automated testing was introduced to overcome the above limitations of manual testing.

What is Testup

Testup is a visual test automation tool to test web applications, and it is one of the easiest test automation tools on the web. It is developed and maintained by Thetaris GmbH, a group of software professionals. Further, Testup is a no-code automation testing tool. Hence, you do not need to write any code to test the software.

Why do we need to use Testup

Testup testing helps to reduce the amount of time required for manual testing dramatically. It also provides an easy transition from a manual test to an automated test using the visual test editor. 

Some automated testing tools require from the users advanced knowledge and experience of test automation. This is one of the biggest advantages of Testup. To use Testup there is no advanced knowledge or experience of test automation required. Learning Testup is easy and intuitive. If needed, a lot of learning resources can be found on the official website of Testup.

Testup can capture more bugs than with manual effort and also capture more bugs before they hit their users. It is also easy to set-up the test environment and takes just a few minutes as Testup is one of the best user-friendly testing tools.

Further, the ability to test complex test scenarios with ease, unlimited interactive test executions and high accuracy of test results are some other benefits of using Testup.

Testup uses the black-box testing approach. Therefore, a user without any coding knowledge can use this testing tool for software testing.

Many WordPress developers face a problem of website crashes after a plugin update. Testup provides an excellent solution for this problem by running a test to see if the website is still working as earlier after a plugin update. Further, if you are a blog owner who is not a web developer, then you can update your website and verify with Testup, if it is working correctly without the help of the developer or agency.

Considering all the above facts, Testup is an ideal automated testing tool for a user who has little or no experience with test automation.

How to perform a Testup testing

Testup has introduced two plans named “standard” plan and “enterprise” plan. The standard plan is a monthly subscription. But the user can subscribe for the 7-day-free-trial first in order to get to know the tool. To get started free with Testup, the user can just sign up here. After that, he can start testing by creating projects and tests.

Conclusion

Testup is an excellent test automation tool for users who have never done test automation. Its wide variety of features enables the user to test the software with ease and obtain accurate test results.

We hope you enjoy this article. Happy Testing!

Posted on by — Leave a comment

Testing with a Twist: How Testup Tests Itself

Testup is a frontend test automation system for web pages and front ends. But Testup does not only provide a tool to test front ends it also has a nice web front end itself. To assure that it remains like that, it should sound natural that we use our own software to test itself. Sounds twisted? Well…

Let’s first review some of the common challenges in UI testing.

Ideal Reality
Low Redundancy A generic redesign of the application requires redundant test updates at many locations.
Full automation Distinguishing design changes from design failures requires human intervention.
Reproducibility UI is particularly prone to inconsistencies when interactions occur at super human speed and internal states are not yet prepared.
Transparent State The internal state of server components are not accessible after the test ran through.
Locatablility Failure that surface end of a complex interaction cannot easily be attributed to a single failed feature or step.
Speed Certain features can only be reached after lengthy preparation phase. Causing for lengthy warm up periods.

Currently we have 11 UI tests. Each of the tests is focused on a specific aspect of the test application. Each of these tests activates several features that are attributed to that feature. Some tests share common predecessors that prepare the software in a stable state that can be used as a basis for more advanced features that require a filled data base.

The following picture gives you an outline of the relevant screens from each test. Please note that some predecessor tests have multiple continuations. For each of these continuations the entire predecessor must be rerun from scratch to ensure a clean state.

How are we doing on our own ideals?

  • Low redundancy
    We have shared predecessors that are 100% reused. E.g. the login process is only defined once and reused everywhere.
    Once the tests veer into different directions from a common predecessor there is no more sharing of test steps. The tests must be defined with the least possible overlap of accessed features.
  • Full automation
    In the good case tests run fully autonomously from start to end. What if it breaks? First, all test checks occur on the graphical representation of the application. Hence, it is easy for a human to assess the difference between expected and observed state. Second, tests will try to recover from the visual change and present the differences to the user who then accepts the change. In our dreams it would be smart enough to close a cookie banner, but we are not there yet.
  • Reproducibility
    It is not trivial to define tests that run consistently under variations of uncontrollable variables, e.g. server load, network latency, or expected changes in displayed calendars and times. This is certainly the hardest part and it totally relies on the usability of the software to make tracing and fixing issues as much fun as possible. (It cannot be explained until you use the software)
  • Transparent state
    Our approach is fully graphical and as such we might see less of an internal state (e.g. the DOM). However, we do record the entire screen sequence and can thus highlight any early deviations from the base line. Hence, it is usually possible to navigate quickly to the earliest indication of an incorrect internal state.
  • Locatability
    If tests were written to just replay recorded screen interactions it would be difficult to fail exactly at the point where the erroneous feature was executed. Instead we define tests such that they contain frequent checks and assertions. Adding an assertion is as easy as drawing a rectangle around the area you think should be graphically stable.
  • Speed
    Let’s face it. UI tests are not unit tests. Our tests are currently taking our 30 minutes of CPU time and it’s growing. That’s why our service comes with access to a cluster that can run tests massively in parallel.

To summarize this post with a (biased) view on our own software I have to say that we are quite happy. We have made progress on most of our set goals. In terms of usability I am convinced we have already surpassed most competition. If you haven’t done yet please do sign up and share your views.

“Testing with a Twist” is a series of articles about testing with Testup. Up to now the following articles were published:

Posted on by — Leave a comment

Testup – The Developers choice for Software Testing

Unlike manual testing, automated testing executes test cases using automated tools such as Selenium. However, using automated tools is not always easy and can be a challenge for a single developer due to many reasons.

If you are a developer who has struggled with automated testing tools like Selenium, then you have come to the right place. In this article, we are going to introduce a simple automated testing tool called Testup that helps you to perform test automation with ease.

Let’s begin by finding out the reasons why developers struggled with Selenium!

Why do some developers struggle with Selenium

Even though Selenium is very popular as an automated testing tool in the IT industry, there are several drawbacks involved in the Selenium testing tool. As a result, some users struggled with Selenium. 

The following list below shows some of the drawbacks of Selenium.

  1. The complexity of Selenium
    Selenium is not just a single tool but a suite of software which consists of Selenium IDE, Selenium RC, Selenium Web Driver and Selenium Grid. As a result, using Selenium can be difficult if you don’t understand the tool properly before starting testing using Selenium. 
  2. Lack of coding skills
    Like many other automated testing tools, Selenium also requires some advanced programming skills. Therefore, the tester needs to be familiar with at least one of the supported programming languages for Selenium such as Java, Python, PHP, etc. to write test scripts. 
  3. Not easy to learn
    For a beginner, Selenium is one of the hard testing tools to learn. It requires lots of time, programming knowledge and coding skills and also great resources to learn Selenium. 
  4. Difficult setup
    Setting up the Selenium testing environment is not easy. 
  5. Time consuming
    The amount of time required to prepare test cases is high.
  6. Weak image testing
    Selenium provides less support for image testing.

What is Testup 

Testup is a visible test automated tool to test web applications, and it was developed and maintained by Thetaris GmbH, a group of software professionals. Further, it does not need any coding for testing as it is a no-code automated testing tool. 

Why Testup

The Testup automated testing tool helps to overcome the complexity involved in automated testing. The following list below shows some of the benefits of Testup when compared to other automated testing tools such as Selenium.

  1. Simplicity
    Tesup can be considered as one of the simple automated testing tools in the IT industry. Further, it is a useful tool for someone new to test automation.
  2. Easy use
    Easy to set-up the test environment as Testup is a user-friendly testing tool.
  3. Fast test recording
    The time required to record a test can be dramatically reduced by using the no-code editor of Testup.
  4. No coding required
    The ability to perform testing without writing any programming code. Therefore, the lack of programming knowledge is not a barrier to using Testup.
  5. Easy to learn
    Learning Testup is easy, and it requires less amount of time and resources. Further, all learning materials can be found on the Testup website.
  6. Handles complex test cases
    Testup allows the tester to test complex test scenarios with ease.
  7. Speeds up the workflows
    The visual editor helps to refactor a broken test in a few seconds. Further, it allows unprecedented speed-ups in the complete development workflow.
  8. Easy understandable tests
    Readability of tests is unique due to the visual editor of Testup.
  9. Creates robust tests
    The unique anchor image technology allows the user to create robust tests.

Considering all the above facts, Testup is an ideal automated testing tool even for a user who has struggled with automated testing tools such as Selenium.  

How to perform a Testup testing

Before creating a project, you need to get a licence. You can start with the professional licence, that costs only 1€ for the first month. The registration process is simple and straightforward.

After completing the registration process, the user can create a new project and a new test.

Next, the user can record a test to test the software. Recording a test may involve steps such as selecting an anchor area, mouse-clicking, entering text to search input fields or swiping.

Finally, the tester can exit the editor by pressing the “Done” button and click on the “Run” button to run the test.

If the test passes, Congratulations! Otherwise, try to fix the test by editing it. There is a vier, too, that helps you to find the problem.

Visit Testup documentation if you need help.

Conclusion

In a nutshell, Testup is an excellent automated testing tool for a user who has struggled with automated testing tools like Selenium. Its wide variety of features enable the user to test the software with ease and obtain accurate test results.

We hope this article has helped to overcome the barriers of test automation. Happy Testing!

Posted on by — Leave a comment

Testing with a Twist: The Testup Build Pipeline

How do you write software that tests itself? We at Testup are asking us this question every day, because we are writing software that allows everybody to test their software. And that obviously includes ourselves. One of our main missions is to write software that we love to use for testing our own software. Sounds twisted? Well, ….

Let’s first look at the attributes that we love and who is the hero in that discipline.

AttributeHero technologyIdeal
Fast responseStatic code analysisGet feedback as you type
Full historyStack trace
(Log file)		See everything that happened in run up to the failure
Transparent stateDebuggerSee all variable values at the time of failure
Production like environmentDocker,
Infrastructure as code		Quick reproduction of deployment settings
Smart assessmentsHuman
AI		Make informed decisions about the validity of the observed behavior

How can we achieve all these qualities at once? Unfortunately there is no hammer that hits all nails, at least not all at once. Let’s look at our build pipeline and see how we are doing.

There are three different environments to run the tests:

  • Build environment
    • Triggered with every commit (or locally on request)
    • Has access to only the artefacts of one repository at a time
    • Should finish within few minutes at most
  • Integration environment
    • Replicates all components of the productive environment
    • Mocks/disables external components like payment, messaging, …
    • Automated tests finish within an hour or less
  • Staging environment:
    • Usable environment for humans to interact
    • Final validity check for new features
    • Should be completed within a day

Below you can see the the build process from first commit up to production release. There are only two manual steps:

  • Pull request: Manual inspection of code quality
  • Acceptance test: Manual inspection of the feature’s usability before release

We will cover all aspects of the build pipeline in future posts. Next to come: “Automated UI tests”

“Testing with a Twist” is a series of articles about testing with Testup. Up to now the following articles were published:

Posted on by — Leave a comment

Why Testup is the Best tool for a QA Team

In this article, we are going to discuss an automated testing tool called Testup. If you have a team for automated testing and dedicated QA engineers and testers, this tool will be much easier to use.

Let’s begin!

Introduction

There are several automated testing tools available in the market such as Selenium, Appium, QMetry Automation Studio, etc. But to use some of the automated testing tools, the user needs some advanced knowledge and experience in automation testing. However, Testup is different from most of the other automated testing tools. 

What is Testup 

Testup is a visual test automated tool to test web applications, and it is one of the easy test automated tools on the web. It was developed and maintained by Thetaris GmbH, a group of software professionals. 

Furthermore Testup automated testing tool uses the black-box testing approach that means the user does not have any knowledge of the internal implementation and also does not see the source code of the software. In short: Testup is an easy to use no-code test automation tool!

Advantages of Testup

Testup automated testing tool helps to overcome the complexity involved in automated testing. Further, it has lots of advantages compared to other automated testing tools. 

Testup can be considered as one of the simple automated testing tools in the IT industry. Testup test environment is easy to set-up and it takes just a few minutes as it is one of the best user-friendly testing tools. The ability to perform testing without writing any programming code helps someone who lacks programming knowledge also to use Testup. 

The visual approach of Testup enables manual testers to step into the role of a QA automation engineer. Learning Testup is easy, and all the required learning resources can be found on the official website of Testup. The visual editor helps to refactor a broken test in a few seconds, and it allows unprecedented speed-ups in the complete development workflow. Further, the readability of tests is unique due to the visual editor of Testup.

Testup can manage hundreds of tests and capture more bugs than possible with manual effort and also capture more bugs before they hit their users. Unlimited interactive test executions, the ability to test complex test scenarios with ease are some other benefits of using Testup.

The unique anchor image technology allows the user to create robust tests. Testup testing helps to reduce the amount of time and human resources required for manual testing dramatically. Accuracy of test results is also high as there is no or little possibility of human error.

Many WordPress developers face a problem of website crashes after a plugin update. Testup provides an excellent solution for this problem by running a test to see if the website is still working as earlier after a plugin update. If the test passes, the developer can update the plugin with confidence. Further, if you are a blog owner who is not a web developer, then you can update your website and verify it is working correctly with Testup without the help of the developer or agency.

Testup is not only beneficial for the individual tester but the team also. It makes communication between team members much easier. Further, all members of the team can easily understand the Testup concept, and there are different aspects of Testup for different roles of the team members. It makes the testing task easy while increasing efficiency and effectiveness of testing.

Considering all the above facts, Testup is an ideal automated testing tool for any type of user who is interested in test automation. 

How to perform a Testup testing

Before creating a project, you need to get a licence. The professional licence costs only 1€ for the first month. Enough time to get to know the software.

After completing the registration process, the user can create a new project and a new test.

Next, the user can record a test to test the software. Recording a test may involve steps such as selecting an anchor area, mouse-clicking, entering text to search input fields or swiping.

Finally, the tester can exit the editor by pressing the “Done” button and click on the “Run” button to run the test.

If the test passes, Congratulations! Otherwise, try to fix the test by editing it. There is a vier, too, that helps you to find the problem.

Visit Testup documentation if you need help.

Conclusion

In a nutshell, Testup is an excellent automated testing tool for any type of tester, from a beginner to an expert. Its wide range of features such as simplicity, user-friendliness and ability to test without coding enables the user to test the software with ease and obtain accurate test results.

We hope you enjoy this article.

Happy Testing!

Posted on by — Leave a comment

Our Journey to No-Code Test Automation

For the last 14 years, I have been working on tools for developers at Thetaris. My Co-founder Stefan and I developed, we developed all kinds of tools, from Excel plugins, financial mathematics libraries, test suites, apps to a full-fledged IDE for our own programming language ThetaML. During this work, we found that many developers in our community struggle with maintaining their applications. The developers fix a bug or implement a new requirement and suddenly, a seemingly unrelated feature of the application breaks.

Unit Testing is part of the solution

There should be an easy answer: testing. From the large projects we completed at Thetaris, we know that Unit-Tests are part of the solution. They are quick and can be used to automatically stop bugs from entering the code case. But they are not sufficient. The applications still break.

Manual testing is mandatory

Manual testing is an expensive and time-consuming solution. However, it works. The testers identify not only functional issues, but also UX glitches and text typos. Consequently, testers need the focus of attention. The testers should be enabled to maintain their own automated tests. Over the years, we tried many different approaches to reach this goal.

Selenium-Python, Cucumber

I thought, why not make my testers to QA engineers. We created a Python library which uses a simple language such that test cases are easy to understand and easy to extend. In other projects we used Cucumber, a simple language to make test cases easy to understand. Both approaches delivered the same result: A QA engineer is required for the language definition and testers can run tests. In some cases, tester can add test themselves. But, usually, they need a developer again who changes some of the application code.

Visual Testing

Why would that be? Why can testers not just test the software without knowledge of the internal workings? I thought that a manual tester uses the User Interface, only. This must be completely sufficient to write automated tests using the UI. Using my computer vision experiences, I created a prototype in Python to provide a proof of concept for a visual test automation. It worked surprisingly well. No inner knowledge of the application was necessary anymore.

Testing in Cloud

Another main issue testers face when running automated tests is the constantly changing infrastructure: E.g. Google Chrome gets an update and the Selenium Driver breaks. The connection of the native app using Appium on the local desk is another constant source of issues. Working in the cloud with Browserstack for native devices and Mabl for websites shows that testing belongs in the cloud. Testers are more productive and happier when they do not have to care about the test system infrastructure.

Simplicity

The last missing piece to a wide acceptance of test tools by testers is simplicity. I am lucky that my Co-founder is a UX nerd. During the development of our own test platform, Stefan included iteration after iteration to minimize the required user interaction for test recording. Minimizing the decisions a user has to make, we are also minimizing errors the user can do. We constantly collect feedback from new users of our application and simplify it further.

Putting it all together: Testup

Now, we have put it all together in the Testup project: Visual Testing using Computer Vision, Infrastructure in the Cloud and Simplicity by No-Code test creation. The effort is fruitful: Test are easy to create, easy to ready and easy to change. The next step is the version 0.9 which we will release in a few days with an open beta. We are looking forward to your feedback!