Prove a change

Overview#

Agents now write more code than people can review, and an AI reviewer is one more model's opinion. MLola changes the question a person answers. Instead of “is this diff right?”, the person answers “is this what we want?” once, when the change request is approved. After that, every change is held to that approved intent by checks that use no model, and each change comes with a proof: a signed record of which criteria held, with their evidence, and the short list of things a person still has to look at.

git clone https://github.com/mlolahq/browser-runtime && cd browser-runtime && npm install
npm run demo:prove     # before → the agent's first attempt → the fix, in about 30 seconds

In the demo, the agent's first attempt passes its tests and shows “Saved!”, yet the proof stays not proven: the API shows the new name was never stored. The fix turns that criterion from ❌ to ✅.

1. The contract#

A contract (mlola.contract/1) is the change request, written so a machine can check it. Each acceptance criterion says what must hold and how it is proven, at the lowest layer that can prove it.

{
  "format": "mlola.contract/1",
  "id": "CR-123",
  "title": "Users can change their display name",
  "version": 2,
  "approvedBy": ["product owner", "tech lead"],
  "risk": "medium",
  "baseUrl": "http://127.0.0.1:4320",
  "acceptance": [
    { "id": "AC-1", "text": "A display name is trimmed, and an empty or over-long one is refused",
      "proof": { "layer": "test", "report": "../reports/junit.xml",
                 "cases": ["display name > trims surrounding spaces"],
                 "files": ["../test/rules.test.mjs"] } },
    { "id": "AC-2", "text": "Saving a new name on the profile page confirms it",
      "proof": { "layer": "browser", "url": "/profile",
                 "steps": [{ "fill": "Display name", "with": "Ada Lovelace" }, { "click": "Save profile" }],
                 "expect": { "kind": "text_visible", "text": "Saved!" } } },
    { "id": "AC-3", "text": "The new name is stored, not only shown",
      "proof": { "layer": "api", "get": "/api/profile", "bodyContains": "Ada Lovelace" } },
    { "id": "AC-5", "text": "The confirmation wording is approved by the brand team",
      "proof": { "layer": "manual", "how": "the brand team reviews the AC-2 screenshot" } }
  ],
  "invariants": [
    { "id": "INV-1", "text": "The service reports healthy",
      "proof": { "layer": "api", "get": "/api/health", "status": 200 } }
  ]
}

• Paths (report, files) are relative to the contract file. files names the files that hold the test cases: if a change edits them, the change has written its own proof, and the proof says so.
• API checks are GETs, never another method. auth: {"env": "STAGING_TOKEN"} sends a bearer token from the environment; the token is never recorded.
• Browser checks play fill, click and select by the label a person sees, with no selectors. MLola never types into a secret field; start signed in with --storage-state instead.
• Invariants are behaviour that must keep holding (regressions, guarded pages, health), proven the same way.
• risk: "high" means a person always reviews the intent and the evidence, even when everything is proven.
• mlola browser contract check <file> validates a contract and prints its digest and how much of it a machine can check.

2. Proving a change#

mlola browser prove contracts/CR-123.json \
  --base-url http://localhost:3000 \
  --base origin/main \
  --baseline base-proof/proof.json \
  --out proof --summary "$GITHUB_STEP_SUMMARY"

--base <ref> compares the change with its base branch. It tells whether the contract is the approved one (unchanged, changed, new, or approved when its digest is passed in --approved), and whether the change edited the tests that prove it.

--baseline <proof> is a proof of the same contract made on the base branch. Each criterion then shows its verdict before and after: ❌ → ✅ shows the change did it. A criterion that already held before the change is flagged: it may test nothing new.

3. The proof#

proof.json (mlola.proof/1) holds the contract's id, digest and status; the changed files, commits and CI job; and, for every criterion, its verdict before and after, its evidence and its layer's record: test cases and the report's hash, the API answer's status and body hash, or the browser steps with a full receipt of the page. Screenshots are bound to the proof by their SHA-256. Response bodies are not kept, and secrets are masked.

Instead of the whole diff, the proof lists only what a machine cannot settle:

• the intent, when the contract is new or changed;
• the tests, when the change edited the tests that prove it;
• UAT, for manual criteria;
• a criterion that already held before the change;
• everything, when the contract's risk is high.

Auto-merge is true only when the change is proven and that list is empty. The proof is sealed by a digest and signed with ed25519 when a key is configured (mlola browser receipt keygen, or MLOLA_RECEIPT_KEY in CI).

4. Checking a proof#

Anyone can check a proof offline, without a browser, a network or a model:

mlola browser receipt check proof/proof.json --contract contracts/CR-123.json --trust ed25519:3f…

The check confirms integrity, the signature, the contract and the screenshots. It also judges every criterion again from what the proof recorded, then recomputes the verdict, the review list and auto-merge. A proof that was edited and re-sealed, or issued by a modified tool, is contradicted by its own record.

Exit 0: sound and proven. 1: sound, not proven. 4: not sound.

5. In CI#

name: prove
on: [pull_request]

jobs:
  prove:
    runs-on: ubuntu-latest            # Google Chrome is preinstalled
    env:
      MLOLA_RECEIPT_KEY: ${{ secrets.MLOLA_RECEIPT_KEY }}   # signs the proof
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }      # the base branch, for --base
      - uses: actions/setup-node@v4
        with: { node-version: 22 }
      - run: npm ci
      - run: npm test -- --reporter=junit --outputFile=reports/junit.xml
      - name: Start the app
        run: |
          npm run start &
          npx -y wait-on@8 http://localhost:3000
      - name: Prove the change
        run: |
          for contract in contracts/*.json; do
            npx -y -p @mlola/browser-cli@0.15.1 mlola browser prove "$contract" \
              --base-url http://localhost:3000 --base origin/${{ github.base_ref }} \
              --out "proof/$(basename "$contract" .json)" --summary "$GITHUB_STEP_SUMMARY" || status=1
          done
          exit ${status:-0}
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: proof, path: proof/ }

The job summary shows each contract's verdict, its criteria with their evidence, and what is left for a person. To post it on the pull request, feed mlola browser receipt show proof/CR-123/proof.json --markdown to a comment action.

6. For agents#

The MCP tool browser_prove runs a contract from the directory the operator configured (MLOLA_MCP_CONTRACTS), against the agent's dev server. The agent can prove its change before it says the change is done, and keep fixing the code until it is proven. The agent names a contract; it cannot hand in its own or point at a file elsewhere.

What a proof does not claim#

• A proof is as good as its contract. It shows the change does what the contract says, not that the contract says everything. That is why the contract is the thing a person reviews, and why manual criteria exist.
• A test layer reads a report; it does not run the tests. Signing in CI ties the proof to the job that wrote the report.
• A browser script plays labels, not intent. If the page changes its labels, the step fails and says which; a person decides whether the change or the contract is wrong.
• Checks run against the environment you point them at. MLola sends only GETs to the API, but a browser step can click a button that changes data: use a test or preview environment with test data.