# AGENTS.md — stream-droid

Guide for AI agents (and humans) working in this repo. `CLAUDE.md` is a symlink
to this file, so both are always identical. Read this before making changes.

## What this is

**stream-droid** streams a running Android **emulator or device** into the
browser and drives it (tap / swipe / type / keys), for humans and AI agents —
the Android analogue of [serve-sim](https://github.com/EvanBacon/serve-sim). A
React UI shows the live screen with a sidebar to list/boot AVDs; a bun server
bridges the browser to `adb` (and, for emulators, the emulator gRPC API).

It's a **sketch**, not production: one client per device assumed, and
**loopback-bound by default** (control needs the tunnel token, or a local request).

## Stack & tooling

- **Runtime:** [bun](https://bun.sh) ≥ 1.3.11 (pinned in `.bun-version`) **or
  node ≥ 20**. The server is TypeScript run directly — under bun natively, or
  under node via `tsx` (a dependency). `bin/stream-droid.mjs` is the published
  entry and picks the runtime (`typeof Bun`). **Fully bun-optional** — server,
  skill helpers, and the build all run under node or bun. The client bundle is
  built by `scripts/build-client.mjs` (esbuild; a small plugin wraps jmuxer's UMD
  so esbuild can extract its default export — see the script), CSS via
  `npx @tailwindcss/cli`. `ensureAssetsBuilt` builds both on first run under the
  current runtime; the npm package ships them prebuilt so running never builds.
- **Language:** TypeScript, `strict`, **no `any`** (enforced by oxlint).
- **Server deps:** `ws`, `@grpc/grpc-js` + `@grpc/proto-loader`, `ts-pattern`,
  `localtunnel`, `cloudflared` (managed tunnel binary), `qrcode`.
- **Client:** React 19 + `react-dom`, **Tailwind v4** (`@tailwindcss/cli`),
  **tailwind-variants** (`tv`), `jmuxer` (H.264→MSE). Bundled by `bun build`.
- **Lint/format:** [OXC](https://oxc.rs) — `oxlint` (`.oxlintrc.json`) + `oxfmt`
  (`.oxfmtrc.json`). **Not** ESLint/Prettier.

## Commands

```bash
bun install
bun start            # build css+client, then run the server (opens browser)
bun run build        # build:css + build:client
bun run lint         # oxlint src
bun run format       # oxfmt src
bun run check        # oxlint + oxfmt --check + tsc --noEmit   ← CI gate
bun run typecheck    # tsc --noEmit
bun test             # unit tests (bun:test) in __tests__/ folders
```

**Always run `bun run check` before considering a change done.** It must exit 0
(lint clean, formatted, type-clean). Run `bun run format` to auto-fix style.

Run the server directly during dev: `bun run src/server.ts [name] [flags]`
(see `bun run src/server.ts -h`). Use `-d` / `--headless` in scripts/tests so it
doesn't pop a browser tab.

## Layout (where things live)

```
bin/stream-droid.mjs  published bin: launches the server under bun (native) or node (tsx)
src/
  server.ts        entry: match(config.mode) → help/list/kill/log/serve wiring only
  config.ts        parsed CLI/env → `config` singleton; isAuthorized(); fail()
  log.ts           leveled logger; quiet by default — only error() prints without -v/--verbose
  adb.ts           adbFor(serial), resolveSerial/targetSerial, deviceSize, sendPoster
  controllers.ts   Control/Incoming types; adb/scrcpy/grpc controllers; pickController
  httpServer.ts    static assets + /api/{state,start,stop,health,tunnel,shutdown,apps,launch,hierarchy} (ts-pattern routes)
  wsServer.ts      per-connection: stream frames out, route control in
  commands.ts      -h help · -a list · --kill · -l log · --tunnel
  tunnel.ts        tunnel lifecycle: open/stop/info; cloudflared (default, via npm bin) or localtunnel
  lifecycle.ts     preflight, first-run asset build, boot target, openBrowser
  emulator.ts      list AVDs / running devices, boot (headless), tooling checks
  apps.ts          list packages / launch app / foreground app (adb; pure parsers)
  semantic.ts      uiautomator hierarchy dump + parse + findElement
  capture/
    types.ts       CaptureHandle / EmulatorInput / CaptureOptions contracts
    select.ts      startCapture() — pick backend from config
    screenrecord.ts / scrcpy.ts / grpc.ts   the three backends
    scrcpyControl.ts   scrcpy v4.1 control-message binary encoder
    scrcpyServer.ts    resolve/auto-download + SHA-256-verify the scrcpy jar
  grpc/            emulator_controller.proto, emulatorClient.ts, discovery.ts
  ui/              React app: App.tsx, hooks (useDeviceStream/useKeyboard),
                   api.ts, token.ts, types.ts, styles.css, components/*
  types/           ambient d.ts for untyped deps (jmuxer, localtunnel)
public/            index.html (shell) + built client.js / app.css (gitignored)
skills/            the `stream-droid` plugin — four agent skills (namespaced /stream-droid:<skill>)
  drive/           see & act loop; owns the shared scripts
    SKILL.md       the loop + quick reference
    scripts/       ensure-server.mjs (start/health) · drive.mjs (control) · check.mjs (prereqs) — plain ESM, bun OR node ≥ 18
    references/    cli, input-control, semantic-layer, browser-ui, agent-skill
  emulators/       list / boot (headless) / kill AVDs  (SKILL.md + references/)
  apps/            list / launch / foreground apps      (reuses drive's scripts)
  share/           public link + QR, view-only vs control
```

Skills reuse `drive`'s scripts via `$CLAUDE_PLUGIN_ROOT`, so they ship together.
Server protocol/pipeline internals stay in the code + `docs/`, not the skill
surface. Don't hand-bump skill/plugin versions on a change — the **published plugin
version** (`SKILL.md` + `.claude-plugin/*.json`) advances only on a stable release
(`version:release`, run by publish-stable); CI's `version:check` guards it.

## Conventions (follow these)

- **Config is a singleton.** All CLI/env parsing lives in `src/config.ts` and is
  read via `config.X`. Never re-parse `process.argv` elsewhere. Add a new flag
  there; if it takes a value, add it to `VALUE_FLAGS`.
- **Logging via `src/log.ts`, not `console.*`.** Use `log.info/warn/error` or a
  scoped `logger('scope')`. The logger is **quiet by default**: `info`, `warn`,
  and `debug` print only with `-v`/`--verbose` (which also timestamps); only
  `error` always prints. Put per-frame/control detail in `.debug()`, and
  gate raw child-process output on `config.VERBOSE`. Don't log the server's own
  HTTP access lines — the client polls `/api/state` every 3 s, so per-request logs
  just flood the terminal even under `-v`. Standalone command output
  (`-h`/`-a`/`-l`/`--kill`/tunnel QR) uses `console.*` directly so it always shows.
- **`ts-pattern` for dispatch & discriminated unions.** Use `match(...).with(...)
  .exhaustive()` for command dispatch, the control-message switch in each
  controller, capture selection, and platform branching — not `switch`/if-chains.
  It's used in the client too (e.g. `Screen.tsx` renders per `ConnState`); the
  ~6 KB it adds to the bundle is an accepted trade for exhaustive, readable
  state rendering.
- **All classNames go through `tailwind-variants` (`tv`)** — `tv({ base })`,
  `tv({ slots })`, or `tv({ variants })`. There is no `cn` helper; don't add one.
- **React:** function components + hooks only. Imperative stream/WS/jMuxer logic
  lives in `useDeviceStream`; components stay declarative. Import React types by
  name (`import { useRef, type RefObject } from 'react'`) — do **not** use the
  `React.X` global namespace (breaks under `verbatimModuleSyntax`). Client is
  built by `scripts/build-client.mjs` (esbuild, under node or bun; a plugin wraps
  jmuxer's UMD so its default export resolves) — set `NODE_ENV=production` via
  `define`, as that script does.
- **Imports use explicit `.ts`/`.tsx` extensions** (bundler resolution).
- **Types for untyped deps** go in `src/types/*.d.ts` (module declarations).
- **No `any`.** Type dynamic gRPC/proto surfaces with narrow interfaces (see
  `grpc/emulatorClient.ts`).
- **Hoist helpers that don't capture locals** (oxlint `consistent-function-scoping`).
- Style: single quotes, semicolons, 2-space, ~110 col, trailing commas (oxfmt).

## Architecture notes / gotchas (don't regress these)

- **Capture backends** (`--capture`): `screenrecord` (default, any device, H.264),
  `scrcpy` (uses the **v4.1** jar — older versions crash on Android 14+; uses
  `raw_stream=true`, H.264; the jar is auto-downloaded + SHA-256-verified into
  `~/.cache/stream-droid` by `scrcpyServer.ts` if `--scrcpy-server` isn't given),
  `grpc` (**emulator-only**, PNG frames via
  `streamScreenshot`, no jar/adb for capture). Codec is `h264` for the first two,
  `png` for gRPC; the client renders `<video>` (jMuxer) vs `<canvas>` accordingly.
- **Input paths** (auto-picked in `pickController`): gRPC RPCs > scrcpy control
  socket > `adb input`. scrcpy opens a **second** socket for control and blocks
  video until it connects — that's why `scrcpy.ts` opens control on video
  *establish*, not on first data (deadlock otherwise).
- **Instant preview (poster).** H.264/MSE is slow & flaky to start from an idle
  screen's single frame (seconds, or never). On connect the server sends one
  `screencap` PNG (h264 modes) and the client sets it as the `<video>`'s
  **`poster`** attribute. Do **not** "fix" this by hiding the video to show a
  canvas — a `display:none` video won't autoplay and playback stalls forever.
- **Network exposure.** The server binds **`127.0.0.1`** by default (`config.HOST`),
  so it isn't reachable from the LAN; tunnels still work (the relay connects over
  loopback). `--host 0.0.0.0` opts into LAN exposure. `config.isRemote(req)` =
  came over the relay (forwarding header) or a non-loopback address;
  `config.canControl(req)` = local operator **or** control token.
- **Link-preview QR (`og:image`).** While a tunnel is open, `serveStatic` injects
  Open Graph tags into `index.html` (absolute URLs, so they can't be baked into the
  static file) pointing at `/og-qr.png` — the same QR as the share panel, as a PNG
  because unfurlers don't render SVG. **That route is deliberately unauthenticated**:
  the unfurler fetching it is an anonymous third party, so `ogQr()` is not host-gated
  the way `tunnelInfo`'s `qr` is. Consequence, accepted by choice so the preview
  always matches the link that was shared: in **control mode** the QR encodes
  `?k=<token>`, so any caller who can reach the tunnel — including a view-only
  viewer — can fetch that image and scan the token out of it, and the preview
  service caches a copy. With no tunnel open the route 404s and no tags are added.
- **Tunnel security.** Sharing is **view-only** by default; control is gated
  by a random token (`config.CONTROL_TOKEN`). Local browser gets `?k=<token>`;
  the shared link carries it only in control mode. **Two permission tiers for a
  remote:** *driving* (WS input, `/api/{launch,apps,hierarchy,health}`) needs
  `canControl` (the token) — a control viewer drives, a view-only viewer only
  watches; *management* (`/api/{start,stop}` device lifecycle, both `/api/tunnel`
  actions, `/api/shutdown`) is **host-only** (`!isRemote`), so no remote — even
  with control — can boot/kill devices or reshare. `/api/state` returns
  `host: !isRemote(req)`; the client hides the device rail + Share from non-host
  viewers. cloudflared prints the URL before the edge routes, so `openTunnel`
  runs `waitReachable()` (polls the public URL until our server answers) before
  reporting the share ready — the link works when it's handed out, not a few
  seconds later. `tunnelInfo.active` also requires `shareUrl`, so a poll mid-open
  never shows a half-ready share. Two ways to open a tunnel,
  both host-only: the CLI (`--tunnel` / `--tunnel-control`), or the app-bar
  **Share** popover → `POST /api/tunnel {action:'start', control:boolean}`
  (`openTunnel(port, control)` sets a per-tunnel `controlMode`; `{action:'stop'}`
  closes it). **Opening any tunnel calls `config.ensureControlToken()` first** —
  a server started without `--tunnel` has no token, so `isAuthorized()` is `true`
  for everyone (fine while loopback-bound); minting one before the relay is live
  keeps a UI-started view-only share (link without `?k=`) genuinely view-only.
  The local operator stays in control (loopback is never `isRemote`). The share
  panel/link/QR and both tunnel actions are gated `!isRemote`. localtunnel is a
  public relay — treat as untrusted.
- **Semantic layer.** `/api/hierarchy` = `uiautomator dump` parsed (decode XML
  entities!). `{type:'tapElement', id|text}` resolves an element center → normal
  tap, so it works across all input backends.
- **gRPC discovery/auth.** The emulator advertises `pid_*.ini`
  (`grpc.port` + `grpc.token`); we send `authorization: Bearer <token>`. Coords
  are device pixels; keys are W3C DOM names (`GoHome`/`GoBack`/`AppSwitch`).
- **Serial vs AVD name.** `--serial`/positional accept either; `resolveSerial`
  and `targetSerial` map by adb serial **or** AVD name (case-insensitive).
- **Headless cleanup on exit.** Emulators this server booted headless are tracked
  in `emulator.ts`'s `bootedHeadless`, and closing the server shouldn't strand
  windowless devices. But that record is keyed by AVD **name** and records intent
  at boot time, so it goes stale — the AVD can be killed and cold-booted *windowed*
  later while its name stays in the set. So the record only picks the candidates
  (anything we never booted is never a candidate); live `ps` state decides.
  `shutdownCandidates()` splits them into `headless` (verified still windowless →
  closed silently) and `windowed` (on screen now → must be confirmed), and prunes
  names that stopped running. Where `ps` is unavailable (Windows) it can't verify,
  so it trusts the record as before. `SIGINT`/`SIGTERM` (`server.ts` `cleanupOnExit`)
  **prompts** before closing a windowed one and defaults to leaving it — a second
  Ctrl-C exits immediately, and with no TTY it never closes them. `/api/shutdown`
  closes only the verified-windowless ones; there's no way to ask over HTTP.
  Emulators the server never booted are never touched either way. The UI defaults
  new boots to headless unless a windowed emulator is already running.

## How to extend

- **New CLI flag:** add to `config.ts` (and `VALUE_FLAGS` if it takes a value);
  document in `commands.ts` `printHelp()` and the README flags table.
- **New capture backend:** implement `CaptureHandle` in `src/capture/xxx.ts`,
  add a branch in `capture/select.ts`, and set its codec expectation.
- **New control message:** extend `Control` in `controllers.ts`, handle it in all
  three controllers (exhaustive `match`), and in the client `send`/`Screen`.
- **New API route:** add to `httpServer.ts` (route on `path`, read `?k=` off
  `url` for auth via `isAuthorized`).
- **New React component:** `src/ui/components/`, classNames via `tv`.

## Verifying changes

Unit tests run with **`bun test`** (`bun:test`); they live in `__tests__/` folders
next to the code (e.g. `src/capture/__tests__/`) and are excluded from the npm
tarball. Coverage is light and focused on pure/verifiable logic (e.g. the scrcpy
jar download + checksum resolver) — add tests there when you touch that kind of
code. Most behaviour still needs a **real emulator**:

- Start it headless: `bun run src/server.ts -d` (add `CAPTURE=grpc` etc).
- Stream/control check: connect a WS client to
  `ws://localhost:3200/?serial=emulator-5554`, count binary frames, send a
  `{type:'tap'|'key'...}`, and confirm device state via
  `adb -s <serial> shell dumpsys window | grep mCurrentFocus`.
- UI/responsive/render: drive a browser (e.g. Playwright) — check the LIVE
  indicator, the mobile drawer, and time-to-first-frame.
- Then `bun run check`.

## Housekeeping

- **Don't commit** build artifacts (`public/client.js`, `public/app.css`) or the
  `scrcpy-server-*` jar — all gitignored.
- macOS `sed` is BSD (no `\b`); prefer the editing tools over shell `sed`.
- Keep the README in sync when you change flags, commands, or layout.