7 September 2026 · Maintenance

Your agent's rules file rots the moment you stop rewriting it by hand

Every team running Claude Code or Cursor writes a CLAUDE.md or a set of skills, and every team watches the agent quietly violate them within a week — not out of malice, just because a rule was too vague or the codebase moved past it. Rewriting the file by hand after every correction doesn't scale past one contributor. The fix is to mine your own session transcripts for the corrections you keep making, cluster them into patterns, and let that process draft the rule update as a diff you approve or dismiss — not one you write from a blank page.

Concretely: an extractor that watches for correction events in past sessions, a clustering step that groups the same complaint together instead of surfacing it 40 times, and a diff generator that proposes the smallest rule change that would have prevented the pattern. A human still approves every line — the point is to stop being the one who first notices the pattern.

How it works

A correction event is any point in a session where the human's next message reads as an override of what the agent just did — "no, we don't do it that way," "revert that," "actually use X." If you use Claude Code, that's already sitting on disk, unaggregated: every session is a JSONL file under ~/.claude/projects/, one line per turn, and every Edit tool call the agent made is in there with the exact file_path/old_string/new_string it used. Nothing to instrument — just read what's already written. (This is the CLI's current on-disk format, not a documented API — check it still looks like this before relying on it. Cursor keeps its own local history in a different shape; the extraction step below is specific to Claude Code, the clustering and diffing steps after it are not.)

# reads Claude Code's own session logs directly, no instrumentation needed
import json
from pathlib import Path

SESSIONS = Path.home() / ".claude" / "projects"
CORRECTION_CUES = ("no,", "don't", "revert", "not like that", "actually use")

def edits_in(turns):
    for t in turns:
        if t.get("type") != "assistant":
            continue
        for block in t["message"].get("content", []):
            if block.get("type") == "tool_use" and block.get("name") == "Edit":
                yield block["input"]  # {file_path, old_string, new_string, ...}

def find_corrections():
    for log in SESSIONS.glob("*/*.jsonl"):
        turns = [json.loads(l) for l in log.read_text().splitlines() if l.strip()]
        for i, turn in enumerate(turns):
            if turn.get("type") != "user":
                continue
            text = str(turn.get("message", {}).get("content", ""))
            if not any(cue in text.lower() for cue in CORRECTION_CUES):
                continue
            prior_edits = list(edits_in(turns[max(0, i - 4):i]))
            if prior_edits:
                yield {"session": log.stem, "edit": prior_edits[-1], "human_said": text}

Run that over a few weeks of sessions and you get a pile of correction events, most of them one-offs. The useful signal is the ones that recur — three different sessions where the agent used any despite a "no any types" rule, four sessions where it skipped a null check on an external API response. Cluster on embedding similarity between the human's correction text and the file path touched, and anything above a size threshold (3–4 occurrences is a reasonable floor) becomes a candidate.

For each candidate cluster, the generator drafts the smallest patch to the rules file that addresses the pattern — not a rewrite, a diff:

--- CLAUDE.md
+++ CLAUDE.md
@@ Coding standards
- Avoid `any`, prefer explicit types.
+ Avoid `any`, prefer explicit types. For external API responses,
+ validate with a schema (zod/io-ts) before narrowing — do not
+ narrow with an inline type assertion.
+ (Seen 4x this month: agent narrowed with `as ApiResponse` and
+  skipped runtime validation. Sessions: a91f3, b204c, c7712, d0561.)

That comment line citing session IDs is the part worth keeping even after the diff is approved — it's the audit trail for why the rule exists, which is usually the first thing that gets lost when a team edits CLAUDE.md by hand under time pressure.

The part most people skip: proving the diff worked

Detecting the pattern and proposing the diff is the easy 80%. The part that actually justifies running this as a recurring job is closing the loop — checking whether an approved diff reduced anything, instead of trusting that a better-worded rule must be a better rule. Four numbers are enough to track, before and after each approved diff:

• Correction rate — how often the same rule gets manually overridden again in the following weeks. A diff that doesn't move this number didn't address the actual cause.
• Time-to-recurrence — days until the same pattern resurfaces. Fast recurrence means the diff patched a symptom, not the rule the agent actually reads that way.
• Tokens/turns per task in the affected area — a rule that sticks usually also cuts the back-and-forth needed to get a correct answer.
• Diff approval rate over time — if it drifts down across cycles, the clustering itself has started proposing lower-quality diffs, and that's worth catching before the backlog of dismissed suggestions trains everyone to stop reading them.

None of these needs a dashboard to start — a spreadsheet updated once a week per approved diff is enough to tell a real fix from a rewording that felt better.

Where it pays, and where it doesn't

The honest downsides. This only sees corrections made inside a Claude Code session — a fix made later in a PR review, in an editor after the session ended, or in a completely different tool, leaves no trace here unless you also diff git history against session timestamps. Clustering on too few sessions manufactures patterns out of noise — a size floor of 3–4 is a guess, not a proof, and it's worth checking a handful of proposed clusters by hand before trusting the threshold. A diff generator will happily encode one reviewer's personal style as a permanent rule if nobody separates "this is objectively wrong" from "this isn't how I'd have written it." And a rules file that grows only by accretion — patch after patch, never pruned — ends up as unreadable as the drift it was meant to fix; someone still has to periodically read the whole file and cut what's no longer true.

The alternative isn't "no maintenance" — it's the same drift, just noticed later, usually in a production incident instead of a review comment.