Posted on Leave a comment

Test an iOS App

Testing your own Android or iOS app requires to create a test with a mobile device and adding a reference to the uploaded file.

Upload file

  1. Get your <API Key> from Profile Page (Details)
  2. Upload a .zip or .tar.gz file containing your compressed .app bundle.

    Your .app bundle must represent a simulator build of your app. After running in iOS Simulator via Xcode, look in ~/Library/Developer/Xcode/DerivedData/<project-name>/Build/Products/Debug-iphonesimulator/.

    Alternatively, you may run xcodebuild -sdk iphonesimulator (if you use .xcodeproj) or xcodebuild -sdk iphonesimulator -workspace Sample.xcworkspace/ -scheme <your-scheme> -configuration Debug (if you use .xcworkspace) in your project directory, then zip the .app bundle in build/Debug-iphonesimulator/.
  3. Upload ZIP using the following curl commands
curl "https://app.testup.io/uploader/new?ApiKey=<API_KEY>&platform=ios" -F "file=@app-release.zip"

This curl command returns a JSON with <publicKey>.

You can update an existing APK using this <publicKey> with:

curl "https://app.testup.io/uploader/update/<publicKey>?ApiKey=<API_KEY>&platform=ios" -F "file=@app-release.zip"

Create and link test

  1. Create a new Test
  2. Select Mobile System
  3. Select an iOS device you need
  4. Create a varible by clicking “+” in Advances Section
  5. AppetizeAppPublicKey = <publicKey>

Create your Test

Click “Edit” on top-right of the “Test Settings” screen

Posted on Leave a comment

Test an Android App

Testing your own Android or iOS app requires to create a test with a mobile device and adding a reference to the uploaded file.

Upload file

  1. Get your <API Key> from Profile Page (Details)
  2. Go to your developer console and create an APK, call it e.g. “app-release.apk”:
    After your app is built, either via Android Studio or by running the command ./gradlew assembleDebug in your project directory, look in <project-name>/<module-name>/build/outputs/apk/
  3. Upload APK using the following curl commands
curl "https://app.testup.io/uploader/new?ApiKey=<API_KEY>&platform=android" -F "file=@app-release.apk"

This curl command returns a JSON with <publicKey>.

You can update an existing APK with:

curl "https://app.testup.io/uploader/update/<publicKey>?ApiKey=<API_KEY>&platform=android" -F "file=@app-release.apk"

Create and link test

  1. Create a new Test
  2. Select Mobile System
  3. Select an Android device you need
  4. Create a varible by clicking “+” in Advances Section
  5. AppetizeAppPublicKey = <publicKey>

Create your Test

Click “Edit” on top-right of the “Test Settings” screen

Posted on

How to evaluate a PHP commands to manipulate variables and clipboard (Experimental feature)

Add Control->EVAL to test

EVAL argument is a PHP 5 expression. See https://php-legacy-docs.zend.com/manual/php5/en/index on how to write php
The result of the expression is returned to the CLIPBOARD

Working with Data

Data from Data Tab in Editor can be read from the test to the clipboard.

E.g. setting

FirstName: Barbara
LastName: Mustermann
Age: 26

Read Data

EVAL $FirstName

Reads the value of the FirstName (“Barbara”) into the CLIPBOARD. Note that there is a “$” in front of the variable name using the EVAL command.

Manipulate Data

EVAL $FirstName." ".$LastName

Concatenates $FirstName, ” “, $LastName using the “.” as usual in PHP and returns the result to the CLIPBOARD.

Compute with Data

EVAL $Age + 10

Adds 10 to the variable Age. In the example with Age = 26, this command returns 36 to the CLIPBOARD.

PHP Examples

Measure time between steps

EVAL microtime(true)
:
EVAL microtime(true) -  $CLIPBOARD

Returns the time passes in seconds to CLIPBOARD.

Call PHP for TimeStamp

EVAL date_format(new DateTime(), "d.m.Y H:i:s");

Returns a timestamp to CLIPBOARD

Call Java for TimeStamp

EVAL new Java("java.util.Date")->toString();

Returns a timestamp to CLIPBOARD

Manipulate strings

EVAL substr($CLIPBOARD, 0,strlen($CLIPBOARD)-5)

Removes the last 4 characters from CLIPBOARD

Set and Get values from data store

EVAL get("result")."\r\n".$CLIPBOARD)

Concatenates via “.” the string in the data store”result” with a newline (“\r\n”) and CLIPBOARD contents.


EVAL set("result")

Sets the value of the data store “result” to the value of the CLIPBOARD.

EVAL set("result", get("result") ."\r\n". $CLIPBOARD)

Performs the concatenation of the setting of the data store with a single line of code.

Posted on Leave a comment

Create random Emails

To create a random Email use button. The values are stored in buffer – clipboard content – and then can be pasted into your test.

Insert random Email into a test.

Make sure you are in mode.

  1. Make an entry field active and enter your Email address.
  2. Insert a click where the random number is to be inserted.
  3. Switch to mode.
  4. Click .
  5. Type in a PHP code to create a random number: rand(1,100000000)
    A random number will be inserted in clipboard.
  6. Click on to add the created random number to your email.

Posted on Leave a comment

Notifications

Get notifications for automated run

Make sure you open a schedule that runs automatically.

  1. Open tab “Notifications” .
  2. Check the box “Send Notifications”.
  3. Choose “failed runs” or “every run” in drop down menu.
  4. Check the box “Notify on Slack”.
  5. Insert a unique URL created with Incoming Webhooks in Slack (ex. https://hooks.slack.com/services/xxx/xxxx).
    Please notice: you need to open Slack settings to configure Incoming Webhooks in Slack and to create a unique Url. Check Slack installation instructions here https://api.slack.com/messaging/webhooks.

Posted on Leave a comment

Using schedules for automated runs

Run your tests automatically

Create a schedule to run one or several tests automatically at a certain time. A schedule can be run hourly, daily or weekly.

Setup a Schedule:

  1. Open an execution page.
  2. Click .
  3. Add tests to your new schedule.
  4. Open a schedule and rename it.
  5. Check the box “Run automatically”.
  6. Enter the time and frequency when Testup should run your tests.
Posted on Leave a comment

How to integrate tests into Gitlab CI/CD pipelines

This tutorial shows how you can easily run all your tests from a Gitlab pipeline. We assume that your current pipeline is already able to build and deploy your software. This tutorial contains the following steps to complete this task:

  1. Get your ApiKey from Testup to authenticate your Gitlab pipeline
  2. Insert the ApiKey into a secure space as a Gitlab variable.
  3. Extract the Id of your schedule
  4. Setup the Gitlab pipeline to run your tests
  5. Optional: Make temporary changes to the settings in your test

1) Get your ApiKey from Testup

Go to your Testup start page and click on the “Profile” tab. Find your ApiKey und the section “Api Keys”

2) Provide your ApiKey to Gitlab

Since the api key gives full access to your testup settings it should be provided to the Gitlab pipelines in a secure way. To do this open Gitlab and follow these steps:

  1. Open the project settings
  2. Open the CI/CD section
  3. Expand the Variables section
  4. Press “Add variable”

Once you successfully opened the add variable dialog you can provide your details as follows:

  1. Change your variable’s name to “TESTUP_APIKEY”
  2. Paste the Testup ApiKey into the value field
  3. Uncheck the “Protect variable” box (or alternatively protect your pipelines that should run your tests)
  4. Press “Add variable” to complete the setup

3) Find your Testup schedule Id

In this step you must find the numeric Id of the schedule running your tests. For this step you navigate to your project in the browser. From there you open (or create) the schedule, such that you see a domain that it has the form http://app.testup.io/schedule/<scheduleId>?… Keep that schedule Id ready as you will need it in the next step.

4) Setup your Gitlab Pipeline

We assume that you already have a running pipeline that builds and deploys your project. Testup can only run your tests with access a deployed version of your software. Therefore, you need to add the end-to-end test as a final stage in your build pipeline. Go to your stages section and add e2e-test. Your file .gitlab-ci.yml will probably look something like this:

stages:          # List of stages for jobs, and their order of execution
  - build
  - unit-test
  - deploy
  - e2e-test     # Add this stage here

As a next step you add your end-to-end test step at the end of your pipline description. Your new pipeline step can be inserted as shown below. Don’t forget to provide the correct schedule Id.

Testup:   # Run this job after deployment completed successfully
  stage: e2e-test
  image: curlimages/curl
  variables:
    SCHEDULE_ID: <YourScheduleId>
  script:
    - URL="https://app.testup.io/cicd/schedule/$SCHEDULE_ID/run/$CI_JOB_ID"
    - curl $URL -H Authorization:\ ApiKey-v1\ ${TESTUP_APIKEY} -s --retry 12
    - curl $URL -H Authorization:\ ApiKey-v1\ ${TESTUP_APIKEY} -s --fail

How exactly does this step work technically? First, it starts with an image that provides the “curl” command. Then it builds the url that triggers the start of the pipeline. This url contains the schedule id to run as well as the Gitlab job id to distinguish update requests from new runs. Following these preparations two curl commands are issued. The first calls Testup and retries until the test is either marked as failed or passed. Until ready the endpoint returns a 504 timeout code along with some early debug information. This first curl also makes sure that your pipeline’s debug messages contain useful information and a link to the corresponding resource in Testup. The second curl is necessary to make the pipeline fail if the tests failed.

Once your pipeline is set up you will see a debug message in your pipeline that looks something like this:

4) Optional: Provide additional settings to your test

Very often it is necessary to run your pipeline tests with other values than the ones used in interactive editing. Common cases are temporary domains or changing parameters for users, passwords etc. You can provide additional parameters in the message body of the curl. It is possible to replace urls and text contents that occur in your test. Your pipeline would then look like this:

Testup:   # Run this job after deployment completed successfully
  stage: e2e-test
  image: curlimages/curl
  variables:
    SCHEDULE_ID: <YourScheduleId>
  script:
    - URL="https://app.testup.io/cicd/schedule/$SCHEDULE_ID/run/$CI_JOB_ID"
    - curl $URL 
      -H "Authorization:\ ApiKey-v1\ ${TESTUP_APIKEY}"
      -H "Content-Type: application/json"
      -s --retry 12
      --data '{
       "message": "Execution title shown in testup.io",
       "variables": [{"name":"some_var", "value":"some value"}],
       "urlMap":[{
        "old":"https://stage.example.com",
        "new":"https://qa.example.com",
        "regex":false
       }],
       "textMap":[{
        "old":"OriginalValue",
        "new":"NewValue",
        "regex":false
       }]d
      }' 
    - curl $URL -H Authorization:\ ApiKey-v1\ ${TESTUP_APIKEY} -s --fail