Posted on Leave a comment

Running Tests from a Script

There are many use cases where you want to start a test from a script. A common scenario is a regular execution from a server cron job or within a CI/CD build pipeline.

The first thing you need is your personal API-key. In the Testup Web-App (https://app.testup.io) you will find the API Key under the tab “Profiles”:

Now you can use the following APIs in a shell, for example:

Run a test.

curl -X POST "https://app.testup.io/api/v1/project/run-all/<Project ID>?testsToReplay=<Test Case ID>"     
         -H "accept: */*" 
         -H "Content-Type: application/json"              
         -H "Authorization: ApiKey-v1 <APIKEY>"              
         -d "{\"message\":\"New run by curl\",\"threads\":1}"

where 

<APIKEY> is your API-key (see above),
<Project ID>  is the Project ID of your test (see URL of your project, e.g. 6607334),
<Test Case ID>  is the Test Case ID of your test (see URL of your test).
"New run by curl\" is the name of the new execution the command will create. You can change this freely.
"threads\" is the number of parallel test executions. 

This curl returns a JSON with the execution id <id>. Using this <id> you can get the status of the results:

curl -X GET "https://app.testup.io/api/v1/execution/<id>/details"              
     -H "accept: */*"              
     -H "Authorization: ApiKey-v1 <APIKEY>"

If you want to run all test cases at once, you can use

curl -X POST "https://app.testup.io/api/v1/project/run-all/<Project ID>"                
     -H "accept: */*" 
     -H "Content-Type: application/json"                
     -H "Authorization: ApiKey-v1 <APIKEY>"                
     -d "{\"message\":\"New run by curl\",\"threads\":2}"

Posted on Leave a comment

No-Code Test Automation Grabs Center Stage

Test automation is rapidly evolving today. More specifically, no-code test automation siezes the center stage because, with this AI-powered testing platform, non technical members can design and execute test cases to verify new app releases! No-code test automation with apps like TestUp dissolves the dependency between engineers and semi-technical staff in enterprises. Now, any and all business intelligence (BI) members can take charge of verifying elements of an app whose performance is their particular priority. We are now exploring a contrast between low-code automation and truly automated testware. What are the tangible benefits of codeless test automation?

  • Easy to create test cases – even for non technical users
  • Quickly update test cases on app revisions
  • Keep pace with development cycle of CI / CD
  • Reduce QA budget – no engineers needed
  • Build tests across all hardware platforms
  • Users can easily understand each others’ test cases

The groundbreaking benefit of no-code testing is to bring intelligent testing within the capability of all enterprise members, including product owners and business analysts. This reduced dependency on engineers is said to democratize software testing. It makes testing more accessible and less costly. It also makes the dialog of issue resolution much more accessible. The heavy code based regression testing is replaced by a light intuitive interface.

Illustration 1. Shows how easily we can create a no-code test. In this example, search input and results are verified for a UI by the computer vision algorithm.

Another important trend in test automation is hyper automation. Software industry experts widely predict that hyper automation, which entails the combined use of AI, ML, NLP methods as well as RPA – Robotic Process Automation, will rise to a position of common use in coming months. The goal of hyper automation tools ultimately is to replace human intelligence where possible and accelerate those tasks in which human capital is not absolutely necessary. In the field of test automation, experts predict that AI and RPA will also assist human testers for such tasks as  automation of test case reporting and alerts. 

Transcending Coded Test Cases          

While Selenium automation testing owns much of the market because of its comprehensive support by the most popular programming languages such as Python, Ruby, and Java, the design of test cases with Selenium still requires coding skills. Selenium WebDriver and Selenium Grid are still the domain of developers and QA engineers, and business-facing members remain dependent on those engineers to orchestrate test cases in conjunction with the CI / CD environments. Such dependency now vanishes with no-code test automation.

The new generation of codeless testing tools are based on artificial intelligence components including computer vision modeling. Computer vision algorithms can quickly verify page rendering completeness. Using machine learning methods, the new AI-powered test automation tools can also evaluate performance metrics to assist in understanding user experience. And the exceptional advantage is that anyone can create test cases with the intuitive UIs and dashboards. AI-based no-code testers can even send alerts to endpoints using email or Slack when critical issues like authentication credentials appear to be compromised during test execution. No-code test platforms also integrate with CI / CD pipelines more quickly and easily than coded testing.

Rapid Test Case Updates 

In all types of testing, including performance, functional, and regression testing, AI-powered test automation expands previous boundaries by increasing the longevity of test cases. The self-healing test case is a prime example in UI testing with computer vision. AI-based test automation tools are capable of real-time learning by consuming data generated by their own  output at each test run. For example, when a new version release removes a page element expected by a test case, AI test bots can learn to recognize equivalent or alternate elements so that automated browser test cases do not break. The benefits of such AI-driven testing which self-heals at run-time include increased longevity of test cases and reduced QA testing budgets.

Illustration 2. We can run existing codeless tests easily from the dashboard to quickly verify the status of the application under test.

Keeping Pace with DevOps

Code-heavy test automation which requires developers and QA engineers to maintain often results in a testing suite that lags behind the rate of release for new software versions. No-code test automation resolves this problem because test cases can be designed quickly by team members of varying technical knowledge. Intuitive UIs combine with computer vision methods to instantly create test cases in tandem with application development. Bugs can then be identified and reported by all concerned endpoints. Expert testing is no longer limited to QA members. Continuous integration and continuous delivery now flow efficiently and are no longer delayed at the QA stage.

No-Code Tests Reduce Expenses

Startup AI enterprises often need to optimize human capital to reduce operating expenses. Very often this means developers will also do functional and regression testing, and often code the test cases themselves. This is relatively expensive when AI test automation apps can take on the work of QA. Moreover, a feedback loop can evolve when there are not enough fresh eyes on a project. AI can provide the much needed objectivity by way of automating tests and generating its own code. It is also useful to note that many such test bots make their auto-generated code available via editor for revision by qualified team members. But we are now moving into an era when manual coding is obsolete and expensive. No-code testing automation is the budget-friendly QA department enterprises need.

Platform-Independent Testing

Among the most refreshing aspects of no-code automation testing is its total independence from device and OS constraints. Computer vision-based testing in particular is totally portable across mobile, tablet, and laptop UI. Very often, creating test cases is a simple matter of drawing selection boundaries around UI elements under test. Subsequently, any user can click the run button and examine the outcome. In other words, the computer vision algorithm does not care whether its imaging target is running on a phone or a tablet or a desktop. The same outcomes will be reported by the testing tool.

Interoperability Across Staff Domains

The technical jargon of developers and engineers has always been an obstacle which decelerated the work flows in organizations with both tech and non tech staff. The problem extends into the realm of testing because QA is a buffer zone with a lot of interaction between developers and users. This zone has been peppered in recent years with spurious attempts to create testing tools for non technical members. Such attempts included low-code tools or testing languages like Cucumber, which was supposed to be usable by non technical members. It turned out, however, to be another language for developers to learn, and actually increased the overhead of testing and app development! Fortunately, true AI-based no-code testing now rescues all members from this additional tedium and expense by making testing accessible to everyone.

The Future Now of No-Code Test Automation

We are now witnessing the transformation and revival of test automation from drudgery to exciting new potential. The creative new wave of AI in test automation is making this revival a frontier which everyone can partake of. New apps like Testup are leading the charge to fast and efficient testing for tech savvy and non tech members alike.  Of course, the expert knowledge domain of testing will be in an energetic flux for years to come. AI will inevitably evolve to advanced image recognition and DOM element recognition to improve the flexibility of browser and UI testing. Algorithms in design today will do amazing feats like capturing data from test revisions and predicting components of new test cases to further reduce overhead of test case revision. Stay tuned to TestUp as we stay on the front lines of this exciting new frontier!

Posted on Leave a comment

Ultimate Guide to WP-Staging

Websites today are increasingly complex in design. Because of the volume of code and the numerous technologies packed into a single website, an apparently simple change in code can cause an unpredictable outcome in the performance of the site. Making code revisions or other changes directly to a live website creates unacceptable risk, especially when downtime can result in loss of revenue to an Ecommerce enterprise. For this primary reason, a modern website is first developed on a staging site. Then, after testing shows that no new bugs arose from code changes, the staging site is deployed to the live site, also called “production.” WP-Staging is a plugin for WordPress sites which greatly facilitates and simplifies the creation and operation of a staging environment for WordPress sites. 

Benefits of Staging WordPress Sites

Why use a staging environment? Even a modest WordPress site often contains half a million lines of code in PHP, MySQL, jQuery, Angular, React, and much more. The backbone of WordPress sites also contains all the code libraries of the WordPress Codex, a vast resource of functions. The number of potential bugs and performance issues which can arise from a seemingly simple code revision is unlimited and represents an extraordinary risk to production of Ecommerce enterprise. WP-Staging empowers us to sandbox development risk by first revising code on a WordPress staging site. Then, only after QA has tested and verified the new version as bug-free, WP-Staging automates deployment of the new version to live production. The benefits of WP-Staging are truly persuasive:

  • Automatic cloning of existing site to staging site environment
  • Translation of live site paths to staging site paths in code
  • Detection of files which have changed
  • Automatic deployment of new app version to live site
  • Automatic rollback to previous version of code
  • Authentication provides security on staging site

How to Use WP-Staging

The first step in setting up a staging site is to clone the live or production site. In this step, WP-Staging creates a replica of your website on your server in a new location which you designate. In the setup stage, you can choose to copy the production MySQL database and include or exclude tables as required. Or, you can use the live version of your MySQL db with the staging site. As WP-Staging copies files according to your settings, it automatically revises paths in code files to point correctly to the new locations. Cloning is very fast and does not affect the performance of the live site while in progress. Several important options which you can configure for cloning include:

  • Duplicate the MySQL database for use with staging
  • Optionally use live site version of MySQL
  • Exclude specified tables in your MySQL database
  • Designate specific user access and roles

Intuitive Features of WP-Staging 

WP-Staging is a WordPress plug-in which accomoplishes an amazing set of tasks. One of the great developer-oriented features, WP-Staging color codes the new staging site WordPress dashboard to quickly and obviously distinguish it from the production site Admin dashboard. When you first sign in to the staging site and the new Admin Dashboard is bright orange, you know right away this is the staging admin panel, and there is no way to accidentally make design changes to the production site. That’s just one of many awesome intuitive features of WP-Staging.is a WordPress plug-in which accomoplishes an amazing set of tasks.

Another important feature is the automation of tedious tasks. Many WordPress users choose WordPress content management system (CMS) because it reduces the technical overhead significantly. WP-Staging amplifies this by automating many of the complex tasks of manually creating a staging site.

WP-Staging is an ALL-In-One Solution

WP-Staging features both a free version and a paid version. Only the basic function of cloning a website for the purpose of creating a staging site is provided in the free version. All the more advanced features such as migrating a site to multiple domains, and pushing an entire site to production, require the paid version. The pro version is very affordable and under €100 at the time of writing.

The daily workflow of coding changes to a WordPress site is significantly more efficient when using WP-Staging to roll out new versions of your site. WP-Staging eliminates the risk of doing so, while providing a convenient method to roll back to previous versions when needed. Using WP-Staging is very much like having a continuous integration – continuous delivery (CI/CD) engineer on staff for the low cost of a Wp plugin. WP-Staging is a great all-in-one solution for companies in need of low overhead technology and developer solutions!

References

Image captures:

https://athemes.com/tutorials/how-to-setup-a-wordpress-staging-site/

Posted on Leave a comment

Text inputs

To enter and send text inputs to the Test Device:

Make sure the Editor is in mode.

Make sure the entry field on the Device Screen is active.

  1. Click on on Editor menu
  2. Type the text with your keyboard
  3. Press [Enter] on your keyboard to send the input to the Test Device.

Please notice

As an advanced user you can skip pressing .
Just type in the text and press [Enter] to confirm and play the action.

Posted on Leave a comment

Our Journey to No-Code Test Automation

For the last 14 years, I have been working on tools for developers at Thetaris. My Co-founder Stefan and I developed, we developed all kinds of tools, from Excel plugins, financial mathematics libraries, test suites, apps to a full-fledged IDE for our own programming language ThetaML. During this work, we found that many developers in our community struggle with maintaining their applications. The developers fix a bug or implement a new requirement and suddenly, a seemingly unrelated feature of the application breaks.

Unit Testing is part of the solution

There should be an easy answer: testing. From the large projects we completed at Thetaris, we know that Unit-Tests are part of the solution. They are quick and can be used to automatically stop bugs from entering the code case. But they are not sufficient. The applications still break.

Manual testing is mandatory

Manual testing is an expensive and time-consuming solution. However, it works. The testers identify not only functional issues, but also UX glitches and text typos. Consequently, testers need the focus of attention. The testers should be enabled to maintain their own automated tests. Over the years, we tried many different approaches to reach this goal.

Selenium-Python, Cucumber

I thought, why not make my testers to QA engineers. We created a Python library which uses a simple language such that test cases are easy to understand and easy to extend. In other projects we used Cucumber, a simple language to make test cases easy to understand. Both approaches delivered the same result: A QA engineer is required for the language definition and testers can run tests. In some cases, tester can add test themselves. But, usually, they need a developer again who changes some of the application code.

Visual Testing

Why would that be? Why can testers not just test the software without knowledge of the internal workings? I thought that a manual tester uses the User Interface, only. This must be completely sufficient to write automated tests using the UI. Using my computer vision experiences, I created a prototype in Python to provide a proof of concept for a visual test automation. It worked surprisingly well. No inner knowledge of the application was necessary anymore.

Testing in Cloud

Another main issue testers face when running automated tests is the constantly changing infrastructure: E.g. Google Chrome gets an update and the Selenium Driver breaks. The connection of the native app using Appium on the local desk is another constant source of issues. Working in the cloud with Browserstack for native devices and Mabl for websites shows that testing belongs in the cloud. Testers are more productive and happier when they do not have to care about the test system infrastructure.

Simplicity

The last missing piece to a wide acceptance of test tools by testers is simplicity. I am lucky that my Co-founder is a UX nerd. During the development of our own test platform, Stefan included iteration after iteration to minimize the required user interaction for test recording. Minimizing the decisions a user has to make, we are also minimizing errors the user can do. We constantly collect feedback from new users of our application and simplify it further.

Putting it all together: Testup

Now, we have put it all together in the Testup project: Visual Testing using Computer Vision, Infrastructure in the Cloud and Simplicity by No-Code test creation. The effort is fruitful: Test are easy to create, easy to ready and easy to change. The next step is the version 0.9 which we will release in a few days with an open beta. We are looking forward to your feedback!

Posted on Leave a comment

How can I insert a “page-down” command?

Option 1

Just press the key on your keyboard when “Select anchor area or press key” is shown on bottom right.

Then, a new action with a text area appears.

Option 2

After selecting an anchor area you can press the keyboard on the input bar and enter the text into the field.

Option 3

After selecting an anchor area you can click on the scrolling bar in the browser window of your editor and then play the action.

Option 4

After selecting an anchor area you can swipe by holding the mouse button down, where the swipe shall start and release the mouse button where the swipe ends. Click on “Play this” to perform the action on the system under test.

There’s a special site just about scrolling.

Posted on Leave a comment

I cannot continue. I get a red button with an exclamation mark in it. What can I do?

The anchor image on the right cannot be found on the browser on the left. This can mainly two reasons. Let’s go through them.

Issue 1: Your website has changed.

A common reason for the exclamation mark is that your website has changed. Thus, the image which we call anchor area cannot be found on the website. Make sure to mark regions on your website which do not change. Animations or written date stamps are common pit falls. Here is a description how you should choose your anchor area. Now click on the red button and select “Retake image”.

Issue 2: You skipped a few steps in the editor

Reset the system under test by chosing “Play to here“.

If the problem persists, you probably have Issue 1 and need to “Retake Image”