## Ensuring Coding Agents Prove Their Work: A Traceability Pattern
Modern AI coding agents can confidently report “fixed” while leaving critical questions unanswered: Did the right code change? Did the intended behavior actually improve? Without answers, passing builds and weak assertions create a false sense of security. This article presents a traceability pattern—capturing what the agent inspected, changed, ran, and verified—so every “fixed” claim is backed by evidence, not just code diffs.
The core problem transcends UI fixes. Whether it’s a button overflow, a payment branch mismatch, a type-handler split, an auth redirect, or a behavior-changing refactor, the agent must answer:
– Which file owns the concern?
– Which selector, function, or route is the target?
– What exact patch was applied?
– Which checks prove the user’s complaint is resolved?
Model Context Protocol (MCP) and similar agent tooling make it easier to connect AI actions to external systems, but they only help if the run itself leaves a reviewable record. The tutorial below demonstrates a repeatable pattern: reproduce the failure, apply a narrow patch, verify with the same (or equivalent) check, and save a complete run log.
The example targets a pricing card where a “Start your workspace today” button overflows on a mobile viewport. The fix is a small, focused CSS change, but the value lies in the evidence: file reads, a wrong-selector attempt, the applied patch, build output, DOM measurements, and before/after screenshots. This same pattern scales to API contracts, test failures, database migrations, and authentication flows.
By the end, readers will have a reusable review pattern: identify the likely files, isolate the target behavior, run the check that shows the bug, apply a minimal patch, rerun the check, and retain the full run log. The goal is not this specific pricing card—it’s to stop accepting “fixed” without proof.
### The Developer Problem Is Broader Than UI
Consider common agent requests and how easily they can miss the real issue:
– “Fix the checkout bug” might patch the wrong payment flow or the wrong checkout stage.
– “Make this test pass” can weaken assertions instead of fixing behavior.
– “Update the API response” might change a type or schema without updating the handler.
– “Fix auth redirect” could adjust the frontend while the backend returns 401 instead of 302.
– “Clean up this function” may preserve behavior but look like a safe refactor.
In each case, the critical questions remain: What did the model intend, what actually changed, and what check confirms the user’s complaint is resolved? UI work makes these questions visible through browser measurements, but the same discipline applies everywhere.
### The Tool Setup and Evidence Workflow
The Python wrapper script acts as a lightweight orchestrator:
– It exposes narrow tools similar to MCP capabilities (list_files, read_file, apply_patch, run_build, inspect_dom, capture_screenshot).
– It records every model tool request and the corresponding function result in a run log.
– The model drives the process via tool calls; the wrapper validates and executes, preserving evidence.
This approach is compatible with hosted observability (Weights & Biases Weave, LangSmith, Arize Phoenix, Langfuse), but it first ensures the run log contains the right artifacts. Those logs can later be forwarded to external platforms for dashboards, search, and cost tracking.
### Run the Tutorial
Use the companion repository (abduldattijo/coding-agent-run-recorder). It includes:
– ui_fixing_agent.py: the orchestration script
– app/index.html and app/styles.css: the broken pricing card
– app/scripts/: build-check and UI inspection utilities
– outputs/: run logs, diffs, JSON checks, and screenshots
– media/: visual evidence
Requirements: Python 3.10+, Node.js and npm, Playwright with Chromium, and an OPENAI_API_KEY. Run with –mode api or –mode replay to regenerate evidence without a key.
The run produces structured outputs:
– DOM check JSONs capturing measurements and overflow state
– A targeted patch file
– Build output confirming compilation
– Screenshots showing before and after
– A comprehensive run log detailing every model request and function result
### The Browser Check and Evidence
The browser check uses Playwright locators and bounding-box comparisons at a mobile viewport (390×844). It verifies that the button stays within the card, producing numeric overflow and layout data instead of relying on visual inspection alone. This makes the check objective and repeatable.
Evidence before the fix shows:
– cardBox width 354px, buttonBox width 360px, overflowRightPx 29
– buttonInsideCard: false
The model initially tried a wrong selector (.support-link), which the run log and browser check correctly flagged. The targeted patch updates .primary-action to use width: 100%, max-width: 100%, adjusted padding, and safer white-space handling.
After the fix:
– cardBox width 354px, buttonBox width 308px, overflowRightPx 0
– buttonInsideCard: true
The measurement change from 29px overflow to 0px provides unambiguous proof. Screenshots support the measurement, but the JSON check drives the conclusion.
### What the Run Log Shows and Why It Matters
The run log sequences the complete interaction:
– model requests restore_baseline, list_files, read_file, inspect_dom
– model requests apply_patch (wrong selector), inspect_dom (fails)
– model requests apply_patch (targeted fix), run_build, inspect_dom (passes)
Without this log, developers see only the final code. With it, they can audit every step, identify wrong files or selectors, catch skipped checks, and understand why a seemingly valid change failed.
### How to Adapt This Pattern to Your Own Agent
Follow this reusable workflow:
1) Start from a vague request.
2) Read likely files and identify the target behavior.
3) Run the check that exposes the bug.
4) Apply a narrow patch.
5) Rerun the build and the same check.
6) Save the run log, diff, JSON evidence, and screenshots.
Replace inspect_dom with the check that matches your domain—endpoint calls, test outputs, migration checks, or performance metrics. Keep proof aligned with the original request and always retain the run log. This pattern works for UI, API, database, and refactoring tasks, shifting the standard from “looks fixed” to “proven fixed.”
### Conclusion
A coding agent’s value is not just in proposing changes, but in demonstrating why and how a problem is resolved. Traceability—linking user requests to inspected files, applied patches, executed commands, and verification results—creates accountability and enables meaningful review. By instrumenting tool calls, checks, and outcomes, teams can trust agent-driven fixes, reuse review patterns across domains, and evolve practices with hosted observability tools. The goal is simple: no more “fixed” without evidence.
