From zero to a verified artefact: install, point the runtime at a provider, run one goal, read the evidence.
1 · Install
npm install @mlola/browser-runtime
npx @mlola/browser-cli browser doctor # every line should be green
2 · Providers — pick one
mlola browser config init # writes mlola.config.json
# { "providers": {
# "decision": { "provider": "typesafe", "model": "jev-latest" },
# "reasoning": { "provider": "openrouter", "model": "openai/gpt-5.6-luna" } } }
# or stay with environment variables:
export TYPESAFE_API_KEY=... # TypeSafe is the default fast path
export OPENROUTER_API_KEY=... # one key also runs everything: --fast openrouter
3 · Run a goal
npx @mlola/browser-cli browser run \
--goal "Download my latest invoice" \
--url https://billing.example.com/invoices
4 · Read the evidence
# the run prints status, verification and the artefact
npx @mlola/browser-cli browser trace show <runId> # every decision, action and recovery
mlola browser config show # which routes resolved, credentials redacted
Nothing here needs a third-party site until you point `--url` at one. The repository also ships a local fixture and two runnable demos (`npm run demo`); see demos/README.md for the use cases each one proves.
Requirements
Node 22.18 or newer (Node 24 LTS recommended) and a Chromium-based browser. The CLI drives the Chrome you already have by default, and can fall back to a Playwright-managed Chromium.
Check your environment
node --version # v22.18+ or v24.x
npx @mlola/browser-cli doctor
`doctor` checks Node, the fast-path key, the browser, the runtime home and the daemon. Every line should be green except the optional items.
Install
SDK + CLI
npm install @mlola/browser-runtime
npx @mlola/browser-cli browser doctor # no global install needed
# or install the CLI globally:
npm install -g @mlola/browser-cli
mlola browser doctor
The SDK is a normal ESM package; the CLI and the daemon are optional. Daemon: `npm install -g @mlola/browser-daemon` then `mlola-browserd`.
Fast path: Jev API key
Routine decisions are made by the fast path — TypeSafe Jev — which needs an API key. Without a key you can still run everything with the offline baseline (`--provider keyword`), but that is a deterministic demo baseline, not the decision model.
1Create a key at https://typesafe.ai (TypeSafe account → API keys).
2Store it in the environment — pick one of the options below.
3Verify with doctor until the "Jev fast path" line is green.
Option 1 — export in your shell (persists per session)
export TYPESAFE_API_KEY="tsk_..."
mlola browser doctor
Option 5 — config file (recommended, nothing to export)
mlola browser config init # writes mlola.config.json
mlola browser config show # what the file says + what actually resolved
# and if you only have an OpenRouter key, the fast path runs on it too:
export OPENROUTER_API_KEY=...
mlola browser run --fast openrouter --goal "Download my latest invoice" --url https://...
The CLI reads the shell first, then `.env.local`, then `.env` in the working directory — a local file is enough. In GitHub Actions, store the key as a secret and export it in the step.
Overriding the endpoint or model
Variable
Default
Purpose
TYPESAFE_API_KEY
—
Jev credential (required for the fast path)
MLOLA_JEV_BASE_URL
https://api.typesafe.ai
API base URL
MLOLA_JEV_MODEL
jev-latest
Model id sent in the request
MLOLA_JEV_TIMEOUT_MS
20000
Per-request timeout
Choosing the decision provider
Provider selection
mlola browser run --provider typesafe … # fast path (needs a key)
mlola browser run --provider keyword … # offline baseline, no key required
Your first run
Download an invoice
mlola browser doctor
mlola browser run \
--goal "Download my latest invoice" \
--url https://billing.example.com/invoices
The CLI prints every decision and action, then a summary: status, independent verification, artifacts, provider usage and the fast-path share.
Unattended runs (CI, servers) should pass `--no-human --yes`; without a person the runtime stops safely instead of guessing.
Stronger model (optional)
The fast path covers routine steps. A stronger model is used only when it is genuinely needed — and Jev decides that: when a provider is configured the action space gains an ESCALATE operation, and choosing it makes the runtime ask that provider for a decision from the very same candidate list, validated the same way. The same provider also plans long-horizon goals and writes field values that cannot be reused from the goal.
Providers
Provider
Base URL
Key
Example models
openrouter
https://openrouter.ai/api/v1
OPENROUTER_API_KEY
deepseek/deepseek-v4.1-flash, openai/gpt-5.6-luna
openai
https://api.openai.com/v1
OPENAI_API_KEY
gpt-5.6-luna (or any model on your account)
custom
MLOLA_LLM_BASE_URL
MLOLA_LLM_API_KEY
anything OpenAI-compatible
OpenRouter: one key, many models
export OPENROUTER_API_KEY="sk-or-..."
export MLOLA_LLM_MODEL="deepseek/deepseek-v4.1-flash" # or openai/gpt-5.6-luna
mlola browser doctor --check-llm # verify the key, the model and the round trip
mlola browser run --goal "…" --url https://…
Escalation is charged to the planner role and recorded as fallback events, so the run summary separates cheap decisions from expensive ones: planner calls, fast-path share, and the provider-reported cost. The recovery ladder also caps strong-model consultations (two per run) so a model that cannot fix a problem cannot be consulted forever.
Measure the fast path alone
mlola browser run --llm none --goal "…" --url … # no escalation, no planning, no text model
mlola browser trace summary <runId> # planner calls, fast-path share, cost
The stronger model is optional in the truest sense: with no provider configured the runtime plans less, never escalates, and asks a person for a value it cannot reuse — nothing breaks.
Your own Chrome (extension)
The extension backend drives the Chrome you already use, so your authenticated sessions work without copying a profile. It opens its own Agent Window for the tabs it creates, and a user tab is only moved in after you approve the borrow — then it is returned to its original window and index.
Setup
npm run build && npm run build:extension
mlola browser daemon start
# chrome://extensions → Developer mode → Load unpacked → extensions/chrome
# mlola browser extension pair → token copied to your clipboard + the two fields
# extension options → paste ws://127.0.0.1:7717/extension and the token
mlola browser extension status # is a browser attached, and what can it do?
mlola browser run --browser-extension --goal "Download my latest invoice" --url https://…
Capability
Value
Agent Window + tab groups
yes
Explicit tab borrowing
approval first, returned afterwards
Trusted input
no — synthetic DOM input; use Playwright when fidelity matters
Native dialogs
alerts suppressed; confirm/prompt need a person
Console / network
not available
Uploads
file inputs and drop zones (no file chooser)
The loadable extension ships inside `@mlola/backend-chrome` (`node_modules/@mlola/backend-chrome/extensions/chrome`), so a registry install does not need a repository clone. After editing it, press reload in `chrome://extensions` and confirm the version reported by `mlola browser extension status` — Chrome keeps the old service worker alive otherwise.
The fixture site picks a random port; `npm run fixtures -- --port 4311` pins it so every command in this page works as written.
Capabilities are advertised, never faked: the runtime tells you what the attached browser can do (`mlola browser extension status`) and escalates instead of pretending an action worked.
Configuration
Pin both provider roles in `mlola.config.json` (working directory, then the runtime home) instead of exporting anything.
the fast path: which operation and target to act on
reasoning
openrouter | openai | custom | none
ESCALATE, planning and fallback; `planner` is an accepted alias
Each role also accepts `model`, `apiKey`, `apiKeyEnv` and `baseUrl` — that is how Azure-style endpoints and gateway keys are pinned.
Inspect it
mlola browser config init # write the starter file (never overwrites)
mlola browser config show # the file, plus what actually resolved
mlola browser config show --json
Precedence for every field is flag > shell environment > `.env.local`/`.env` > `mlola.config.json`, so a file is a default and never surprises someone who exported a variable. Unknown roles and providers a role cannot use are rejected at startup, with the offending path named.
Use cases and demo
The runtime is for work where looking like it worked is not enough: documents that must be verified, forms that must not leak a secret, and browsers where you are already signed in.
Use case
What proves it
Document download
an oracle-verified artifact with type, name and digest
Decoy-heavy lists
completion criteria pin the exact document (2026-08, not 2026-07)
Forms with secrets
secret fields never enter a snapshot, a decision or a trace
approval or human takeover; DONE is not offered mid-dialog
Debugging and audit
one JSONL trace per run: decisions, actions, verification, recovery
LLM optional
Jev decides when to escalate; the strong model only picks runtime-owned options
Deterministic CI
scripted decisions against the same fixture: red means the runtime changed
Run the demo
npm install && npm run build
npm run demo # headless Playwright + your own Chrome
npm run demo -- invoice # headless only
npm run demo -- extension # your own Chrome only (daemon + loaded extension)
mlola browser config llm # which provider, model and key it will use
The demo runs against the local fixture site with the real decision path and exits non-zero when a case that should pass did not. Credentials come from the same places the CLI uses: the shell, .env.local, .env.
Drive your own Chrome through the loaded extension
--headless
Run without a visible window
--profile <dir>
Persistent browser profile
--cdp <url>
Attach to an existing Chrome (e.g. http://127.0.0.1:9222)
--policy <file.json>
Policy configuration file
--stop-before "…, …"
Stop at the requested boundary instead of doing it
--max-steps <n>
Step budget for this run
--screenshot off|on-failure|always
Screenshot policy (default on-failure)
--yes
Auto-approve approval-required actions
--no-human
Disable human help (unattended)
--json / --quiet
Machine-readable result / summary only
Task file & boundaries
A task JSON file makes a run reproducible: goal, start URL, the success oracle, the stop boundary, budgets, policy and screenshot mode.
invoice-claim.json
{
"goal": "Download my latest invoice, attach it to a new reimbursement claim titled \"August travel\", then stop before submitting",
"startUrl": "https://billing.example.com/invoices",
"successCriteria": { "criteria": { "kind": "artifact", "documentType": "invoice" } },
"stopBefore": ["submit claim"],
"budgets": { "maxSteps": 30, "maxCostUsd": 0.5 },
"browser": { "backend": "playwright", "headless": true },
"policy": { "defaults": { "upload": "allow" } },
"screenshot": "on-failure"
}
Run it
mlola browser run invoice-claim.json
mlola browser run --task invoice-claim.json --stop-before "submit claim,payment"
`successCriteria` is the independent oracle: `url_contains`, `url_matches`, `title_contains`, `element_visible`, `text_visible`, `artifact`, `download_completed`, `manual`, plus the `all`, `any` and `not` combinations. Without an oracle, a completion the runtime cannot verify is reported honestly (`passed_with_unverified`) instead of being treated as success.
Policy & permissions
Risk is classified in code from the operation and the target label. Modes: `allow`, `ask` (a person approves), `takeover` (a person performs it), `deny`.
Password, OTP and card fields are a hard takeover floor: they are never typed by the runtime and never enter model state or traces. Site rules cannot lower that floor.
Recipes
Copy-paste patterns for the tasks people actually run. Replace the domains, goals and field names with your own; nothing here is specific to any one customer.
1. Download an invoice, attach it to a claim, stop before submitting
One run handles the whole flow: the runtime downloads the file, registers it as an artifact, opens the portal, signs in, types the literal title, uploads the artifact and stops at the boundary instead of submitting.
billing → portal, stop before submit
mlola browser run --goal 'Download my latest invoice, attach it to a new reimbursement claim titled "August travel", then stop before submitting' --url https://billing.example.com/invoices --stop-before "submit claim" --yes
The uploaded file is the artifact this run downloaded — no path guessing. Without --stop-before the runtime would finish the form, which is exactly what a "draft only" policy must prevent.
2. Sites that need login, MFA or a CAPTCHA
Nothing special to configure: the runtime never types a credential. It hands the step to the person, waits, then re-observes and continues on its own.
Interactive
mlola browser run --goal "Sign in and open my claims list" --url https://portal.example.com
# what you will see:
# human needed (credential)
# Take over the browser, finish this step, then hand control back.
# (press Enter when done — the runtime re-observes and resumes)
Credential fields are not merely gated — they never reach the decision model at all. The backend marks a field as secret from its type, autocomplete, label, placeholder, name or id (including labels such as Kata Sandi or Kode Verifikasi) and the action space leaves it out. Unattended runs (--no-human) stop cleanly instead of guessing; in the SDK the same moment arrives as a takeover-required event.
3. Multi-tab flows
OPEN_TAB is offered only for links that really open a new tab, and SWITCH_TAB appears once more than one tab is open. Tabs the runtime opens are closed again at the end; user tabs are never touched.
Download here, upload there
mlola browser run --goal "Download my latest invoice, then open the reimbursement portal in a new tab, sign in and attach it" --url https://billing.example.com/invoices --stop-before "submit"
4. Literal text vs generated text
Quoted values in the goal are reused verbatim, with no model involved. A text provider is only consulted when a value genuinely has to be written from scratch.
Literal first, generation only when needed
# literal: the quoted value is typed as-is (no model call)
mlola browser run --goal 'Enter "August travel" as the claim title' --url https://portal.example.com/claim
# generation: configure a text provider, otherwise the runtime asks a person
export MLOLA_TEXT_BASE_URL="https://api.openai.com/v1"
export MLOLA_TEXT_API_KEY="sk-..."
export MLOLA_TEXT_MODEL="gpt-4o-mini"
mlola browser run --goal "Write a short description of this month's travel expenses in the notes field" --url https://portal.example.com/claim
On a terminal the CLI answers that question itself: it asks "what should I type?" and types your answer (the trace records the value as user-supplied). Nothing is asked when the goal already contains the value — and exploratory goals such as "search some startups" always ask, because there is no value to reuse and inventing one is not the runtime's job.
5. Scheduled runs (daemon or cron)
The daemon owns runs that outlive the command, so a scheduler only has to fire an HTTP request. A plain cron line works too if you prefer everything in-process.
A policy file turns "please be careful" into something the runtime enforces: two allowed origins, uploads allowed, everything that sends or pays stopped or handed over.
The daemon owns runs that outlive a single CLI command, plus the event stream and the Chrome extension bridge. It binds to 127.0.0.1 only and uses a local token.
Lifecycle
mlola browser daemon start --port 7717
mlola browser daemon status
mlola browser daemon stop
npx mlola-browserd --port 7717 # foreground, when the daemon package is installed
~/.mlola/browser-runtime/
runs/<runId>/
meta.json goal, backend, start time
events.jsonl append-only event log (source of truth)
summary.json final result
artifacts/ downloads + registry.json
screenshots/ only per the screenshot policy
daemon/ token, state, pid
MLOLA_HOME=/path/other # override the runtime home
Inspect a run
mlola browser sessions # daftar run terakhir
mlola browser sessions show r_abc12345 # ringkasan + metrik
mlola browser trace show r_abc12345 --limit 40
mlola browser trace show r_abc12345 --type RECOVERY
mlola browser trace summary r_abc12345 # JSON metrik
Every step records the snapshot fingerprint, the offered action space, decision probabilities, policy outcome, freshness result, execution result, verification, recovery state, provider usage and timings — enough to replay a run without guessing.
Safety model
Typed actions only — no selector, coordinate or JavaScript ever comes from a model.
Risk can only be raised by target context; a model cannot lower it.
Secrets are a takeover floor, never typed or traced.
Instruction-like page content is quarantined, not negotiated.
A stale decision never executes: freshness, visibility and occlusion are re-checked before every interaction.
Unknown-effect actions (submits, uploads) are never retried blindly.
Evals
Run the suites
mlola browser eval --suite core # acceptance suite, fast
mlola browser eval --suite security # injection traps, secrets, destructive actions
mlola browser eval --suite matrix # seeded DOM variations
mlola browser eval --suite stress # large pages, re-render, slow downloads
mlola browser eval --suite all
npx @mlola/browser-evals run --suite core --provider typesafe # the real fast path
Results are written to `packages/evals/results/*.json` with status `passed`, `passed_with_unverified`, `failed` or `blocked`. The suites run in a real Chrome against the local fixture site — never a third-party site.
Troubleshooting
Symptom
Fix
missing Jev API key
Set TYPESAFE_API_KEY (see above). Offline: --provider keyword.
Browser does not launch
Run doctor; it reports the channel it tried and falls back to bundled Chromium. Install Chrome or run npx playwright install chromium.
unsupported capability
The backend does not advertise that operation (for example tab borrowing without the extension). This is intentional, not a bug to work around.
Run stops at a credential step
Expected: secret fields never reach the model and are never typed; a person completes them (or --no-human stops cleanly). Use --yes only for approval-required actions, never for secrets.
The run asks what to type, or hands a step over
The goal has no reusable value for that field. Answer the prompt, quote the value in the goal, or configure a text provider (MLOLA_TEXT_*). Exploratory goals always ask by design.
Costs are higher than expected
Check planner calls and fast-path share in the trace summary. Escalation is capped at two per run; --llm none disables the stronger model entirely so you can measure the fast path.
stop-before triggered
The goal asked to stop at that boundary; the run completes with "stopped at the requested boundary".
Daemon unreachable
mlola browser daemon status, then stop/start. Stale state files are cleared on stop.
npm install cannot find the package right after a release
The registry serves a new package through a CDN; the packument can 404 for a few minutes. Retry; dist-tags and tarballs are already live.