Scenario Playbooks

Scenarios are JSON files that drive a sequence of device actions. The format stays intentionally simple: plain JSON, explicit step actions, and no extra DSL layer.

Minimal shape

{
  "name": "android-home-smoke",
  "variables": {
    "appPackage": "com.example.app"
  },
  "setup": [
    { "name": "start app", "action": "startApp", "package": "${var:appPackage}" }
  ],
  "steps": [
    { "name": "go home", "action": "keyevent", "code": "KEYCODE_HOME" },
    { "name": "capture screenshot", "action": "takeScreenshot", "label": "android-home-smoke" }
  ],
  "teardown": [
    { "name": "stop app", "action": "forceStop", "package": "${var:appPackage}" }
  ]
}

Lifecycle

setup is optional and runs before main steps
steps is required and is the main execution phase
teardown is optional and still runs after main-step failures

Step results include a phase field so reports can separate setup, main flow, and cleanup.

Template syntax

Scenario strings support lightweight substitution before execution:

${env:NAME} for a required environment variable
${env:NAME|fallback} for an optional variable with fallback
${var:name} for a root variable
${now:HHmmss} for a timestamp fragment

Metadata and calibration

Use the optional metadata block when a scenario is calibrated against a known app, device, or layout. Luotsi can surface non-fatal metadata_warnings if the live run does not match the scenario’s expected context.

This matters most for coordinate-heavy steps where the wrong device or orientation can invalidate taps.

Action families

Interaction: waitVisible, waitElement, waitNotVisible, tapText, tapElement, tapPoint, doubleTapHeaderLogo (preferred), doubleTap (compatibility alias; header logo only), typeText, typePin, keyevent
Waits and assertions: waitLog, waitStep, waitActionReady, resetLog, assertEvent, assertScreenshot, assertTextInputReady, assertBelow, assertAligned, assertAppVersion
App and package: startApp, startUri, forceStop, clear, clearApp, waitForActivity, waitForNotActivity, isAppInstalled, listInstalledPackages, grantPermission, revokePermission
Artifacts and utility: takeScreenshot, captureArtifacts, screenState, sleep

doubleTap is currently a compatibility alias for the header-logo interaction only. If headerLogo: true is omitted, scenario validation rejects the step.

Use waitElement and tapElement when text is too broad for a stable scenario. They accept a nested selector object with camelCase fields:

{
  "name": "tap Files",
  "action": "tapElement",
  "selector": {
    "text": "Files",
    "textMatch": "exact",
    "resourceId": "com.example.app:id/itemTitle",
    "className": "android.widget.TextView"
  }
}

Selector textMatch defaults to contains; contentDescriptionMatch, resourceIdMatch, and classNameMatch default to exact. Add region with left, top, right, and bottom when duplicate elements need a layout constraint.

Defaults and validation

Common omitted values default intentionally, for example timeoutSec to 15, postTapDelayMs to 300, and milliseconds to 1000 for sleep.

Validation rejects unsupported shapes such as:

tapPoint without coordinates or ratios
negative coordinates
typePin with non-digits
startApp with wait: true but no activity
screenshot region hash checks without a full region definition

Use continueOnError: true on a step when the run should continue after a non-usage runtime failure. Luotsi records that step as continued_on_error and keeps the error payload attached for later triage.

When screenshot-first scenarios are the right call

On older Android builds or screens with weak hierarchy output, prefer explicit tapPoint, takeScreenshot, assertScreenshot, and captureArtifacts steps over brittle selector assumptions. Pair those steps with metadata so Luotsi can emit non-fatal warnings when the device or layout no longer matches the scenario’s calibrated target.

Authoring flow

When the starting point is an Android CLI Journey-style intent (a Journey objective; not an Android Intent object), generate a Journey intake handoff first, then promote reviewed behavior into an executable scenario:

luotsi journey-intake init --output journey-intake.json --package <app.id> --device <serial> --write-markdown
luotsi journey-intake validate --file journey-intake.json
luotsi journey-intake draft-scenario --file journey-intake.json --output scenarios/from-journey.json

The intake file is not executable scenario JSON. Use it to record app context, device assumptions, user goals, assertions, unsafe actions, and the review gate before generating or running a Luotsi scenario. Generated intake files point at the hosted luotsi-journey-intake.v1 JSON schema so editors and agent tooling can check the handoff shape from any output directory. The CLI validator performs the same required-field and command guardrail checks as a CI-friendly handoff gate. The draft command turns a valid intake into a review-required evidence skeleton that can be statically validated before humans replace capture checkpoints with explicit waits and assertions.

luotsi scenario-init --file scenarios/smoke.json --name "smoke"
luotsi scenario-validate --path scenarios
luotsi scenario-explain --file scenarios/smoke.json
luotsi run --path scenarios --device <serial>

Runner modes and outputs

Once scenarios exist, the important runner controls are about planning, validation, and artifact shape rather than just execution.

luotsi scenario-list --path scenarios --include-tag smoke
luotsi run --path scenarios --dry-run --include-tag smoke
luotsi run --path scenarios --validate-only --report-json reports/validate.json
luotsi run --path scenarios --device <serial> --events-jsonl run.jsonl --report-json run.json --report-junit run.xml

scenario-list and run --path share the same discovery filters: --include-tag, --exclude-tag, --name, and --action
run --path --dry-run returns the selected scenario plan after filtering and sharding without validating or touching a device host
--validate-only validates the selected files and still writes reports, but it does not create a device host or execute device work
--dry-run and --validate-only are intentionally mutually exclusive
--events-jsonl, --report-json, and --report-junit are the main machine-readable outputs for CI and artifact review
--capture-on failure|never and --attach-artifacts never|on-failure|always control how much runtime evidence gets attached to run outputs

Parallel and lab-aware runs

When a scenario set grows beyond one device or one developer loop, the key runner controls are:

--shard-count, --shard-index, and --shard-strategy for parallel batches under run --path
--claim-device to create a host-side lab lease for the resolved device during the run
--owner <name> and --ttl-sec <seconds> to label that lease and keep its expiry explicit

These controls matter when the same scenario catalog is being validated in CI, distributed across a device pool, or guarded from two operators grabbing the same hardware at once.

Example files

examples/scenarios/android-home-smoke.json
examples/scenarios/android-navigation-smoke.json
examples/scenarios/buggy-controller-live-demo.json