Local Llama agents in the browser

Each example is one short Python file that talks to Ollama’s HTTP API. A GitHub Actions workflow installs Ollama on a CPU runner, pulls llama3.2:1b, runs the examples live, logs every step through the Evidently template, and commits the transcripts and the report back into this site.

Your browser can’t run Llama, but it can run Python. The runner below loads Pyodide (CPython compiled to WebAssembly) and runs the example code unchanged. The model’s replies come from llm.py’s replay backend, which serves the responses recorded in the latest CI run. The tools, the parsing, the JSON validation and the retry loops all run for real. Edit the code and run it again. If you change a prompt, the replay backend returns the closest recorded reply and labels it nearest match.

Latest CI run

examples/… (editable)Ctrl/⌘ + Enter to run

stdout

Press “Run in browser”. The first run downloads Pyodide (about 10 MB) from cdn.jsdelivr.net.

Recorded trace for this example (one row per span)

The four patterns

#	Pattern	What the Python does around the model	Pass check
1	ReAct	parses `Action:` / `Action Input:`, runs `search_docs` or `calculator`, feeds back `Observation:`; cuts any observation the model invents	answer contains `34302`
2	JSON extraction	Ollama `format: "json"`, then a schema check in plain Python; sends the exact errors back for up to 3 attempts	object validates
3	Planner → executor	JSON plan is validated before anything runs; the model writes the arithmetic, the calculator computes it	answer contains `6.5`
4	Self-critique	model scores its own draft as JSON, plus hard checks (≤ 50 words, has an example) it cannot argue with	hard checks pass

All four share llm.py: a standard-library Ollama client, the replay backend, and a Transcript that records every call (used for replay) and every step (turned into OpenTelemetry spans for Evidently). Repo path: toolkit-src/llama-agents/.

# llm.py — the whole live backend is one POST to Ollama
body = {"model": self.model, "messages": messages, "stream": False,
        "options": {"temperature": 0, "seed": 42}}
if fmt is not None:
    body["format"] = fmt  # "json" or a JSON schema -> Ollama constrains decoding
req = urllib.request.Request(f"{self.host}/api/chat", data=json.dumps(body).encode(),
                             headers={"content-type": "application/json"}, method="POST")

# 02_json_extraction.py — validate, then repair with the exact errors
for attempt in range(1, max_attempts + 1):
    reply = llm.chat(messages, fmt="json", name=f"extract-attempt-{attempt}")
    obj = json.loads(reply)
    errors = validate(obj)
    if not errors:
        return obj, True
    messages += [{"role": "assistant", "content": reply},
                 {"role": "user", "content": "Your JSON failed validation:\n- " + "\n- ".join(errors)
                  + "\nReturn the corrected JSON object only."}]

Run them yourself:

cd toolkit-src/llama-agents
LLAMA_BACKEND=mock python examples/01_react_calculator.py      # offline, no Ollama
ollama serve & ollama pull llama3.2:1b
LLAMA_BACKEND=ollama python examples/01_react_calculator.py    # live
python run_all.py --backend ollama --evidently --site ../..    # what CI runs

How a run gets from GitHub Actions to this page

flowchart TB
  T["workflow_dispatch<br/>or repository_dispatch"] --> R["run-agents job<br/>install Ollama + pull model"]
  R --> X["run_all.py --backend ollama<br/>4 examples, live"]
  X --> E["evllama: OTel spans,<br/>Evidently report + tests"]
  E --> C["commit transcripts, report,<br/>_data/agent_runs.json to stable"]
  C --> B["build job: jekyll build<br/>(same steps as deploy.yml)"]
  B --> D["deploy job:<br/>actions/deploy-pages"]
  D --> P["this page: Pyodide replays<br/>the new transcripts"]

The workflow is .github/workflows/llama-agents.yml. It builds and deploys Pages itself because a push made with GITHUB_TOKEN does not trigger other workflows: deploy.yml would never see the results commit. It still commits the results to stable, so the next ordinary deploy keeps them. It also shares deploy.yml’s concurrency group, pages, so the two never deploy over each other. Permissions are read-only by default. Only the job that commits gets contents: write, and only the deploy job gets pages: write and id-token: write.

Trigger it from the CLI:

gh workflow run llama-agents.yml -R sweeden-ttu/profile -f model=llama3.2:1b -f example=all
gh run watch -R sweeden-ttu/profile

Or from anywhere with repository_dispatch. This needs a fine-grained personal access token scoped to this repository only, with Contents: Read and write. Never paste a real token into a page or a commit.

curl -X POST \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer github_pat_XXXXXXXXXXXXXXXXXXXXXXXX" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/repos/sweeden-ttu/profile/dispatches \
  -d '{"event_type":"run-llama-agents","client_payload":{"model":"llama3.2:1b","example":"all"}}'

model must match ^[A-Za-z0-9._:/-]{1,64}$ and example must be all or an example name. The workflow checks both before using them, and it never interpolates them into a shell script. Keep to small models: the runner is a CPU-only VM.

Recent runs

When (UTC)	Model	Backend	Examples	Evidently tests	Run
2026-10-26 07:52:55	`llama3.2:1b`	mock	4/4	16/16	local seed

The Evidently report for the latest run covers every LLM call: output length, JSON validity, refusals, toxicity heuristics and latency, with pass/fail tests. The data shown now comes from the offline seed: scripted, synthetic responses generated with tools/seed_recordings.py. The first workflow run replaces it with real llama3.2:1b output.