Architecture & Codebase Exploration

The capstone is the sum of every prior module: the agent loop from Lab 02, RAG-style retrieval, memory, evals, tracing, and HITL. First the architecture and scope; then the hardest sub-problem — finding the few relevant files in a repo far too large for the context window.

Hiring research is blunt about this: an issue-to-PR coding agent is the single strongest portfolio piece you can build. It exercises everything — planning, retrieval, tool use, sandboxing, test-driven iteration, evaluation, cost accounting, and human oversight — in one artifact that a reviewer can actually run. This module builds it, then converts it into interview capital.

The capstone-tier project
Hiring guides describe exactly one portfolio project as capstone-tier: an autonomous software-development agent that takes a GitHub issue, understands the codebase, implements a fix, writes tests, and opens a PR with human review before merge. That is this module, verbatim. The context that makes the effort worth it: LinkedIn's Jobs on the Rise 2026 ranks AI Engineer the #1 fastest-growing US job title, and the 'agentic AI' skill cluster grew roughly 280% year over year (~90K US postings). The most-requested skills in those postings — Anthropic/OpenAI tool calling and structured outputs, Docker/E2B-style sandboxing, evals, HITL approval before merge — are exactly what this build exercises.
Scope is the senior move
Do not promise general autonomy. 'Handles simple, well-specified bug-fix issues in Python repos under 10k LOC' is an honest, impressive scope — and stating it that precisely is itself a seniority signal. An agent that reliably does a narrow thing beats one that flakily attempts everything. Your limitations doc starts as this one sentence.
IssueGitHubExploreread codePlanapproachImplementedit filesTestrun suitePRHITL gatetests fail → back to implement (bounded retries)human approves the PR — the agent never merges its own work
The end-to-end pipeline: issue in → explore → plan → implement in sandbox → test loop → HITL-gated PR out.
1/7

The six stages

  1. Input: a GitHub issue URL (or a local issue file) — title, body, and any repro steps.
  2. Explore: map the repo, locate the relevant code, and state your understanding of the bug plus a plan. Checkpoint the plan.
  3. Implement: write the fix in a sandboxed workspace — a git worktree or container, never the real tree.
  4. Verify: run the repo's test suite, write at least one new test reproducing the issue (red → green), iterate on failures up to a bounded retry cap.
  5. Deliver: open a draft PR (or produce a patch + PR description) — gated on HITL approval showing the diff, test results, and cost.
  6. Observe & evaluate: full tracing and a per-issue cost report; run across an eval set and report results.

Architecture decisions worth defending

DecisionOptionsSensible default
OrchestrationHand-rolled agent loop vs. a graph framework (e.g. LangGraph)Checkpointed plan either way; a framework earns its keep once you need durable resume across stages
Plan durabilityIn-memory vs. persisted checkpointsPersist — a crash mid-fix shouldn't discard exploration work
SandboxGit worktree vs. containerWorktree for speed and simplicity on trusted repos; container when running untrusted code
Edit strategySearch/replace vs. full-file rewriteSearch/replace by default (cheaper, safer diffs); covered next lesson
Model choiceOne flagship everywhere vs. cheap-explore + strong-repair splitStart with one strong generalist (claude-sonnet-5 or gpt-5.5); once traces show exploration dominates cost, route it to a cheaper model like claude-haiku-4-5. OpenAI's gpt-5.3-codex line is specialized for agentic coding and worth benchmarking for the repair stage
The context window is the binding constraint
A 10k-LOC repo is far larger than any context window. You cannot paste the codebase in. Exploration is therefore a retrieval problem: find the handful of files that matter and feed only those. Get this wrong and everything downstream degrades — the model plans against files it never saw.

Exploration strategy: agentic search over dumping

There are two ways to locate relevant code, and the agentic one usually wins for this task. Embedding-based retrieval (chunk the repo, embed, semantic search on the issue text) is fine for concept-level 'where is auth handled?' queries. But bugs are often about specific symbols, error strings, and call sites — where agentic grep-and-read shines: give the agent tools to search for symbols, list a directory, and read a file, and let it navigate the way a human engineer does. In practice you combine them: semantic search to seed candidates, then grep/read to confirm and expand.

exploration tools — the agent's eyes on the repo
# Colab cell — pure Python, no key needed. Builds a tiny sandbox repo on
# disk so the three tools actually run against real files.
import subprocess, pathlib

REPO = pathlib.Path("sandbox/repo").resolve()   # absolute so containment checks work
REPO.mkdir(parents=True, exist_ok=True)
(REPO / "pricing.py").write_text(
    "def calculate_discount(price, pct):\n"
    "    return price * (pct / 100)\n")
(REPO / "utils.py").write_text(
    "def clamp(x, lo, hi):\n    return max(lo, min(x, hi))\n")

def search_symbol(pattern: str, max_results: int = 40) -> str:
    """Grep the repo for a symbol or error string. Ripgrep if available."""
    try:
        out = subprocess.run(
            ["rg", "-n", "--max-count", "3", pattern, str(REPO)],
            capture_output=True, text=True, timeout=20,
        ).stdout
    except FileNotFoundError:
        out = subprocess.run(
            ["grep", "-rn", pattern, str(REPO)],
            capture_output=True, text=True, timeout=20,
        ).stdout
    lines = out.splitlines()[:max_results]
    return "\n".join(lines) or "no matches"

def list_dir(rel: str = ".") -> str:
    target = (REPO / rel).resolve()
    # Constrain to the repo — never let the agent wander the filesystem.
    if REPO not in target.parents and target != REPO:
        return "error: path escapes the repo"
    entries = sorted(p.name + ("/" if p.is_dir() else "") for p in target.iterdir())
    return "\n".join(entries)

def read_file(rel: str, start: int = 1, end: int = 400) -> str:
    target = (REPO / rel).resolve()
    if REPO not in target.parents and target != REPO:
        return "error: path escapes the repo"
    if not target.is_file():
        return f"error: {rel} is not a file"
    text = target.read_text(errors="replace").splitlines()
    window = text[start - 1:end]
    # Line numbers help the model reference and later edit precisely.
    return "\n".join(f"{i + start:>5}  {ln}" for i, ln in enumerate(window))


# drive the agent's three "eyes" against the sandbox repo:
print("search_symbol('calculate_discount'):\n" + search_symbol("calculate_discount"))
print("\nlist_dir('.'):\n" + list_dir("."))
print("\nread_file('pricing.py'):\n" + read_file("pricing.py"))
These three tools — search, list, read — are enough for an agent to navigate a repo like an engineer: grep an error string, list the module it points to, read the function, follow the call site. The demo runs each against the sandbox repo so you see exactly what the model would receive — a grep hit with a line number, a directory listing, a line-numbered file window. Two safety essentials: every path is resolved and constrained to the repo (no filesystem escape), and reads are windowed so a huge file can't blow the context budget. Return errors as strings so the model can recover, per the Module 1 convention.
Spot the bug
A teammate reviews list_dir / read_file and 'simplifies' the containment check to a plain string comparison, arguing pathlib is overkill for a sandbox that never sees untrusted input:
python
def list_dir(rel: str = ".") -> str:
    target = str((REPO / rel).resolve())
    if not target.startswith(str(REPO)):
        return "error: path escapes the repo"
    ...
the exploration loop producing a checkpointed plan
# Colab cell — run once. Set your key in the 🔑 panel (name it
# ANTHROPIC_API_KEY) or paste it. This is the assembled agent loop: it needs
# the tools from the previous cell wired into EXPLORE_TOOLS (schemas) and a
# run_explore_tool dispatcher, plus a real repo — it's here to read and adapt.
!pip install -q anthropic

import os
try:
    from google.colab import userdata
    os.environ["ANTHROPIC_API_KEY"] = userdata.get("ANTHROPIC_API_KEY")
except Exception:
    from getpass import getpass
    os.environ.setdefault("ANTHROPIC_API_KEY", getpass("Anthropic API key: "))

import json, pathlib
import anthropic

client = anthropic.Anthropic()
PLAN_PATH = pathlib.Path("sandbox/plan.json")

def explore_and_plan(issue_text: str) -> dict:
    system = (
        "You are a senior engineer triaging a bug. Use search_symbol, "
        "list_dir, and read_file to locate the relevant code. Do NOT guess "
        "at file contents — read them. When confident, call record_plan with "
        "the files you will edit, your understanding of the bug, and the fix "
        "approach. Read only what you need; the repo is large."
    )
    messages = [{"role": "user", "content": f"Issue:\n{issue_text}"}]
    while True:
        resp = client.messages.create(
            model="claude-sonnet-5", max_tokens=4096,
            system=system, tools=EXPLORE_TOOLS,   # search/list/read/record_plan
            messages=messages,
        )
        if resp.stop_reason != "tool_use":
            continue
        messages.append({"role": "assistant", "content": resp.content})
        results = []
        plan = None
        for block in resp.content:
            if block.type != "tool_use":
                continue
            if block.name == "record_plan":
                plan = block.input          # {"files":[...], "bug":"...", "fix":"..."}
            else:
                results.append(run_explore_tool(block))
        if plan is not None:
            # Checkpoint: survives a crash so W22's implement stage can resume.
            PLAN_PATH.write_text(json.dumps(plan, indent=2))
            return plan
        messages.append({"role": "user", "content": results})
The plan is a forced structured output (record_plan is a tool schema, per Module 1) so downstream stages get a typed object, not prose. Persisting it to disk is the checkpoint the README requires: exploration is the expensive part, and a crash during implementation should resume from the plan rather than re-explore. The system prompt's 'do NOT guess, read them' instruction is load-bearing — hallucinated file contents are a top failure mode for coding agents.

Why on-demand search wins in practice

It's tempting to reach for infrastructure: chunk the repo once, embed it, keep a semantic index warm, and query it like RAG. In production coding agents this mostly lost to on-demand agentic search (grep/glob/read), for reasons that generalize past this capstone. Freshness: a repo changes every commit; an index is stale the moment someone merges, and staleness in code search is worse than in prose — a stale hit sends the agent to code that no longer exists. Keeping an index current means a background re-embed job, versioning, and a new class of bugs ('index says line 40, file says line 55'). On-demand search reads the live working tree, so it's correct by construction. Infrastructure cost: an index is a service — storage, an embedding pipeline, a latency budget, a thing that pages someone at 3am. rg is a binary that ships with the OS. For a tool a single engineer or a CI job runs, the infra tax often costs more than it saves. Model capability: frontier models got good enough at iterative, targeted search — grep an error string, read the hit, follow the import — that the coordination overhead of maintaining an index stopped paying for itself for exactly this workload: symbol- and error-string-level bug hunts, not broad conceptual questions.

ApproachFreshnessInfra costBest fit
On-demand grep/glob/readAlways current — reads the live treeNone — ships with the OSMost repos; symbol- and error-string-level bug hunts (this capstone)
Pre-built semantic indexStale until re-embeddedEmbedding pipeline + storage + re-index jobsVery large/monorepo scale, or cross-repo conceptual search
Key insight
The honest caveat: at a scale most companies never reach — multi-million-LOC monorepos — a maintained index earns its keep, because even agentic grep chokes on result volume. Pick the retrieval strategy for the repo size and change frequency you actually have, not the one that sounds more sophisticated.

Whiteboard drills

Check yourself
Drill: "Your grep-and-read exploration works great on a 10k-LOC repo. Now point it at a 2M-LOC monorepo with 40 services — search_symbol for a common name returns thousands of hits. What changes?"
Check yourself
Drill: "Walk me through what happens end-to-end when your read_file tool is asked to read a 50,000-line generated file — a lockfile or a minified bundle."
Key takeaways
  • An issue-to-PR agent is the strongest single portfolio piece; it exercises every prior module.
  • Scope narrowly and honestly ('simple bug-fix issues, Python repos <10k LOC') — precise scoping signals seniority.
  • Six stages: input → explore → implement (sandboxed) → verify → deliver (HITL) → observe/evaluate.
  • The context window can't hold the repo; exploration is a retrieval problem.
  • Prefer agentic grep-and-read (seeded by optional semantic search) — bugs are about specific symbols and call sites.
  • Constrain all file paths to the repo, window reads, and checkpoint the plan so implementation can resume.