Home Original page

GitHub - apptim/apptim-cli-action: Runs performance tests on real devices in minutes ⚡️ - powered by Apptim & AWS.

This action runs Appium tests over Android and iOS in real devices hosted in AWS device Farm. While running tests, it capture performance metrics of the app. Using a dev-friendly YML file, users can define pass-fail criteria to control pipeline workflows.

Key Features

  • Automate app & scripts upload, running and getting artifacts
  • Capture performance data
  • Pass or Fail your pipeline measuring KPIs based on dev-friendly YML file.

Pre requisites

  • Android .apk or iOS .ipa file.
  • The package name of the app.
  • A threshold .yml file for the pass-fail criteria.
  • A config .yml file for the test definition (including test name, app & thresholds path and devices).
  • Apptim & AWS Device Farm credentials.

Usage

workflow.yml Example

Place in a .yml file such as this one in your .github/workflows folder. Refer to the documentation on workflow YAML syntax here.

name: Run Android Tests

on: 
  push: 
    branches:
      - master

jobs: 
  launch-tests: 
    runs-on: ubuntu-latest 
    steps: 
    - name: Checkout code
      uses: actions/checkout@master

    - name: Run tests
      uses: apptim/apptim-cli-action@master
      env: 
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} 
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
        APPTIM_API_KEY: ${{ secrets.APPTIM_API_KEY }}
        CONFIG_PATH: config.yml

Configuration

The following settings must be passed as environment variables as shown in the example. Sensitive information, especially APPTIM_API_KEY,  AWS_ACCESS_KEY_ID  and  AWS_SECRET_ACCESS_KEY, should be set as encrypted secrets — otherwise, they'll be public to anyone browsing your repository's source code and CI logs.

Example of config.yml

name: Startup time test over Android 9
app-file: devicefarm-reference.apk
package-name: "com.amazonaws.devicefarm.android.referenceapp"
test-runner: startup-time
thresholds-file: thresholds.yml
timeout-minutes: 15
device-farm:
        project-arn: "Your Project ARN"  # StartupTime Android Tests
test-devices:
  - device:  # Use this exact device and os version:
    os: "9"

Example of thresholds.yml

startup_time_top:
  moderate:
    operator: ">"
    value: 10000
  warning:
    operator: ">"
    value: 10000
startup_time_avg:
  moderate:
    operator: ">"
    value: 1
  warning:
    operator: ">"
    value: 1