AI Agent Workflows

Luotsi is useful for agent builders because it exposes the same real-device runtime through machine-readable JSON and JSONL that engineers already use interactively. The control loop stays on the host machine, the Android helper stays thin, and you can move from exploration to repeatable runs without swapping stacks.

If you want the smallest working integration first, start with Agent Loop Example. It shows a host process reading inspect JSONL from stdout and writing JSON commands to stdin. If you are still orienting around Luotsi output, read First Five Minutes or run luotsi help output before you build a longer loop.

Pick the right surface

| Surface | Use it when | Primary output | | --- | --- | --- | | inspect | The agent needs to explore the current screen, issue commands, and react to structured deltas. | JSONL session events | | view | A human operator still wants the mirrored device window while the agent consumes stream state and artifacts. | SDL window plus JSONL session events | | discover | The app surface is unknown and the agent needs a bounded state map plus scenario candidates. | Discovery map, JSONL events, and starter scenario JSON | | run | The flow is stable enough to codify as a scenario and run repeatedly in CI or a lab. | One JSON envelope plus artifact root | | replay | The device session is already over and the next question is triage, summarization, or report generation. | JSON or JSONL derived from artifacts |

If you only remember one rule, start with inspect or discover, promote reviewed behavior into run, and use replay whenever you do not need the device attached anymore. The handoff loop is:

command -> structured output -> artifact root -> replay command -> next action

When the agent starts from Android CLI Journey-style intent, use Evidence-Backed Android Journeys to keep natural-language flow authoring separate from Luotsi’s reviewed scenario and replay evidence path.

Install the Luotsi agent skill

This repository ships a repo-local skill at .codex/skills/luotsi-agent. Install or copy that folder into an assistant skill directory, then invoke $luotsi-agent when Codex, Claude Code, or another skill-aware assistant needs Luotsi command selection, artifact-backed replay triage, scenario promotion, or shared-lab/CI-safe workflow guidance.

The skill bundles focused references for first-run workflows, inspect JSONL loops, scenario and CI handoffs, replay/artifact package triage, and Luotsi source ownership. It is meant to teach the agent the product loop before it starts guessing commands:

command -> structured output -> artifact root -> replay command -> next action

Recommended agent loop

1. Prove the device path first

luotsi doctor --device <serial> --fix

This avoids spending tokens and engineer time on missing adb visibility, stale helper state, or missing live-view prerequisites.

2. Explore through `inspect`

luotsi inspect --device <serial>

inspect emits structured JSONL events such as session_started, screen_snapshot, command_result, screen_delta, and session_ended. That makes it the best default surface when an agent needs to reason step by step instead of jumping straight into a scenario file.

The input side is also line-oriented JSON:

{"id":"1","command":"wait_visible","text":"Sign in","text_match":"exact","timeout_sec":15}
{"id":"2","command":"tap_text","text":"Sign in","text_match":"exact","timeout_sec":5}
{"id":"2b","command":"tap_element","text":"Files","text_match":"exact","resource_id":"com.example.app:id/itemTitle","class_name":"android.widget.TextView","timeout_sec":5}
{"id":"3","command":"screenshot","label":"after-sign-in"}
{"id":"4","command":"exit"}

Use text_match: "exact" when a short label could otherwise match a longer string. Add resource_id, content_description, class_name, or region to disambiguate repeated labels; Luotsi reports ambiguous selectors unless you explicitly set allow_ambiguous: true.

3. Promote stable behavior into a scenario

When the app surface is still unknown, run bounded discovery before promotion:

luotsi discover --device <serial> --package <app.id> --budget 5m

Once the flow is no longer exploratory, move reviewed behavior into a JSON scenario so the same device path can run without an interactive loop.

luotsi scenario-validate --path scenarios
luotsi run --path scenarios --device <serial>

Use Scenario Playbooks for the JSON format and Output Envelopes when you need to distinguish one-shot command envelopes from long-lived JSONL sessions.

4. Triage through replay, not through a second device session

luotsi replay packet --artifacts ./artifacts/<run>
luotsi replay packet --artifacts ./artifacts/<run> --check
luotsi replay open --artifacts ./artifacts/<run> --dry-run
luotsi replay summarize --artifacts ./artifacts/<run>
luotsi replay search --artifacts ./artifacts/<run> --contains "error"

The artifact root is the handoff point from live control to post-run analysis. Use replay packet first when an agent or CI job needs the primary failure, recommended next action, 60-second checklist, and a packet that can be validated or passed to another worker. Use replay open --dry-run when a human also needs the browser-free replay front door. Use replay once you already have the evidence you need instead of reconnecting to the device out of habit.

For one-shot envelopes, pick the next command in this order:

data.recommended_next_action_command when a check result exposes the direct continuation command; command-envelope data fields use snake_case
data.recommended_next_action.command
Focused packet evidence such as data.primary_failure.source_command or data.primaryFailure.sourceCommand
Packet checklist commands such as data.triage_checklist[].command or data.triageChecklist[].command
Ordered handoff arrays such as data.recommended_next_steps, data.next_actions, and data.suggested_commands
artifacts.artifact_root as the fallback evidence pointer, then run luotsi replay packet --artifacts <artifact-root>
Command arrays such as data.commands, data.artifact_commands, and data.recommended_commands only when there is no artifact root to packetize

Use replay packet for this fallback because it writes the durable run-summary.json and run-summary.md packet that CI and agents can pass forward. Use luotsi replay packet --artifacts <artifact-root> --check as the validation gate for an existing packet, and use luotsi replay open --artifacts <artifact-root> --dry-run when a human also needs the replay front door response without launching a browser. After picking the next command, carry packet evidenceFiles[] or check-envelope data.evidence_files[] forward as the proof-file context for the focused failure.

Contracts that matter for agent builders

inspect and view are long-lived JSONL sessions.
Most direct commands return one final JSON envelope with schema: "luotsi-command.v1".
Scenario runs keep the normal envelope but also write artifact-rich replay material to disk.
Failure handling is part of the contract: screenshots, hierarchy captures, logcat, metadata, and replay summaries are meant to survive the run.

The public references that matter most are:

Retrieval and planning entry points

Start retrieval from these public surfaces before dropping into source code:

Docs hub: /docs/
Agent-focused entry point: /docs/core-workflows/ai-agent-workflows/
Android CLI Journey entry point: /docs/core-workflows/evidence-backed-android-journeys/
LLM retrieval index: /llms.txt
Command-map reference: CLI Command Groups

When to add `view`

view is not the default just because it looks visual. Use it when the operator needs the live mirror, hotkeys, or observer sharing layer at the same time the agent needs machine-readable session state.

luotsi view --device <serial>

If nobody needs the window, inspect is usually the simpler integration point.