Private, on-device Terraform security review for Chrome
Each milestone has a verifiable exit criterion — the loop’s stop condition. Don’t start a milestone until the previous one’s criterion is met.
| Milestone | Status |
|---|---|
| M0 — Model viability | DONE 2026-07-22 — see m0-results.md |
| M1 — Scanner core | DONE 2026-07-30 — 40 rules, fixture-tested |
| M2 — GitHub integration | DONE 2026-07-30 — see m2-results.md |
| M3 — Report + narration | in progress — parts built, wiring is Queue 2 |
| M4 — Hardening & polish | in progress — download, caching, capability check and options page built; severity tuning and the full audit outstanding |
Question to answer: can Gemma 4 E4B (via @litert-lm/core on WebGPU) produce
useful, non-hallucinated analysis of Terraform code on consumer hardware?
spike/model-eval/ (npm run spike), load the model, paste in the 10 fixture
samples from spike/model-eval/fixtures/ (known-bad Terraform with documented issues)Exit criterion: on ≥8/10 fixtures the model produces a correct, specific explanation with zero fabricated resources/attributes, at usable speed (>10 tok/s). If this fails, the project pivots or stops — decide with the user before M1.
HCL parsing + rule engine with an initial ~15 rule set (public S3/GCS buckets, open security groups 0.0.0.0/0, unencrypted storage/EBS/RDS, hardcoded secrets, missing versioning/logging, overly-broad IAM). Pure TS, no chrome.* imports.
Exit criterion: every rule has known-bad + known-good fixture tests passing; scanner runs against a real-world sample repo checkout and findings are manually spot-checked.
Fine-grained personal access token (read-only) stored in chrome.storage.local;
fetch repo tree + .tf file contents via api.github.com; handle pagination and
100 file repos.
Exit criterion: lists a private repo’s .tf files and runs the scanner on them
end-to-end; no request leaves the allowlist (verified in the DevTools network log).
Met 2026-07-30 via npm run verify-m2 against a real private repo — see
m2-results.md. Verified in the harness rather than the side panel,
because the panel is still a scaffold; the code path exercised is the shipping one
(src/lib/github/client.ts → src/lib/scanner).
Assemble scanner findings into a structured report; feed findings + relevant code snippets to the on-device model for explanation, prioritization, and module-level design commentary. Render in side panel with export (markdown download).
Exit criterion: full run on a real repo produces a report where every stated vulnerability traces back to a rule-engine finding ID; model prose reviewed for fabrications on 3 sample repos.
First-run model download UX (progress, OPFS/Cache API caching), WebGPU capability detection with graceful fallback message, rule severity tuning, options page.
Built: the options page downloads a catalogue model with progress and can delete it or
revoke the host access; the Hugging Face hosts moved to optional_host_permissions so
the install prompt names GitHub alone; the side panel falls back to a downloaded model
and structurally cannot start a download itself. Still open: rule severity tuning, and
the audit below.
dist/)unlimitedStorage is not needed. navigator.storage.estimate() reports a 10.7 GB
quota for the extension origin, against 2.97 GB for the largest catalogue model. So the
install prompt stays at GitHub alone. Residual: Cache API storage without that
permission is best-effort and the browser may evict it under pressure —
navigator.storage.persist() would ask for durability and costs no permission.cache.ts caught the byte-count mismatch, discarded the partial copy and reported it.
This is the guarantee the module exists for and it had never been exercised for real.Cache.put, and narration
end-to-end. The test machine has no WebGPU adapter under this Chromium, so nothing has
yet run a model inside the shipped extension.There is no resume. A 2–3 GB transfer that dies at 95% is retried from zero, and on a
flaky connection may never complete; the user cannot tell “try again” from “this will
never work here”. HTTP Range requests against the partial bytes are the fix.
It is deliberately not a quick one. Resuming means keeping a partial entry on disk, which
is precisely what #download’s discard-on-failure exists to prevent — a partial model
cached under the real key would be served as the whole thing. Any resume design has to
keep partial and complete entries structurally distinguishable, or it trades a slow
download for a corrupt model.
Exit criterion: clean install → report on a fresh machine profile without
developer intervention; extension passes privacy-auditor full audit.