AI coding agents reinventing existing code (dependency blindness)
Dependency blindness is when an AI coding agent writes its own version of a function, class or utility that the standard library, an installed package or the codebase already provides, for example a hand-rolled argument splitter instead of shlex.split. It is about reinventing code that exists elsewhere in the project or its dependencies, not about repetition inside one generated answer. It differs from code hallucination, where the referenced code does not exist at all.
The code may run correctly, so its tests can pass. The knowledge graph describes it as a silent quality regression: it adds code to maintain, duplicates effort and drifts from the conventions of the ecosystem.
At a glance: On SAICA-Bench: No single best reviewer: 3 rows cannot be told apart at 95% on the samples they share (gpt-6-luna, mimo-v2.6-flash and deepseek-v4-flash); the highest J is +90.0% (95% interval +82.0% to +98.0%). Results ↓
- reinvention
- ecosystem duplication
- reinventing the wheel
- reimplementation
- not invented here
- missed code reuse
- duplicate utility
- redundant implementation
- library underuse
How well can dependency blindness be caught?
J 80% or more on our set The agent writes its own version of something the codebase, the standard library or an installed package already provides. No single best reviewer: 3 rows cannot be told apart at 95% on the samples they share (gpt-6-luna, mimo-v2.6-flash and deepseek-v4-flash); the highest J is +90.0% (95% interval +82.0% to +98.0%).
Best reviewer
any ranked row; a tie is shown as a top group
top group of 3: gpt-6-luna, mimo-v2.6-flash and deepseek-v4-flash
Highest J +90.0% (95% interval +82.0% to +98.0%)
Best catalog tool
a tool of the catalog running its own check
No tool of the catalog ran a check of its own on this set.
Shortcut baseline
a rule that understands nothing about the failure
No name from the shown file appears in the added lines
J +38.0%. Read the reviewers against this, not against 0.
What was tested
- Bad samples
- Failure planted on purpose. written by hand by Claude (the coding assistant building this benchmark): one call replaced by a re-implementation. Label: construction; each edit was checked (parses, original call gone, re-implementation present). Clean samples: the same correct agent patch with the original call.
- One sample
- one patch; the two patches of a pair are grouped (intervals resample pairs).
- Samples scored
- 50 patches that re-implement something already available and 50 clean (50% of the set is bad).
- Measure
- J 80% or more on our set.
- Shortcut baseline
- A rule with no understanding of the failure scores J +38.0% here (No name from the shown file appears in the added lines). Read reviewer scores against it, not against 0.
- Tuning
- The clean side was selected on the outcome. 8 of 58 drafted pairs were dropped because the strong models flagged their clean side and a re-read found that the agent's own patch already re-implements something. Missed positives and unflagged clean sides were not re-read, so the step removed false alarms of exactly the models that are ranked. Bound: had the 8 clean sides stayed and been flagged, J would be about +77% for gpt-6-luna instead of +90%, and about +72% for deepseek-v4-flash and +72% for mimo.
All results on this set
Each model asked directly was told which failure to look for and was tested on samples that have that failure or none. Its score is a single-prompt, mode-primed estimate, not a product.
| # | Reviewer | 95% interval | bad samples flagged | clean samples flagged | No verdictanswers left out | False alarmsper 100 clean changes | Precisionif 1% of changes are bad | Precisionif 5% of changes are bad | Precisionif 20% of changes are bad | per 1,000 reviews |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 |
gpt-6-luna, asked directly
Run card
|
+90.0%+82.0% to +98.0% | 96.0%86.5% to 98.9% | 6.0%2.1% to 16.2% | 0 | 6 | 14% | 46% | 80% | $0.24list |
| 2 |
mimo-v2.6-flash, asked directly
Run card
|
+86.0%+76.0% to +94.0% | 88.0%76.2% to 94.4% | 2.0%0.4% to 10.5% | 0 | 2 | 31% | 70% | 92% | $0.32api |
| 3 |
deepseek-v4-flash, asked directly
Run card
|
+86.0%+76.0% to +94.0% | 88.0%76.2% to 94.4% | 2.0%0.4% to 10.5% | 0 | 2 | 31% | 70% | 92% | $0.41api |
| 4 |
grok-4.1-fast (no reasoning), asked directly
Run card
|
+64.0%+50.0% to +76.0% | 94.0%83.8% to 97.9% | 30.0%19.1% to 43.8% | 0 | 30 | 3% | 14% | 44% | $0.53list |
| 5 |
Phi-4-mini, asked directly
Run card
|
−6.4%−24.1% to +11.4% | 42.6%29.5% to 56.7% | 48.9%35.3% to 62.8% | 6 of 100 (incomplete) | 48.9 | 1% | 4% | 18% | $0.19list |
Shown for reference, not ranked
These rows are never counted as best. A shortcut baseline shows what a rule with no understanding of the failure scores on this set; read the reviewers' scores against it.
| Row and why it is not ranked | J95% interval | TPR | FPR | No verdict | Costper 1,000 reviews |
|---|---|---|---|---|---|
|
Rule: a function the patch adds has the same body as one in the shown code (stand-in for sentrux)
Not ranked: baseline rule: shown for reference, never counted as best.
Run card
|
+12.0%+4.0% to +22.0% | 14.0% | 2.0% | 0 | $0 |
|
Shortcut: no name from the shown file appears in the added lines
Not ranked: shortcut baseline: shows what a rule with no understanding of the failure scores on this set.
Where a repository file is shown: flags the patch when none of the functions or classes that file defines is named in the lines the patch adds. Where no file is shown: flags a patch that adds 20 or more lines. It never compares code.
|
+38.0%+24.0% to +52.0% | 56.0% | 18.0% | 0 | $0 |
|
Shortcut: the patch adds 20 or more lines
Not ranked: shortcut baseline: shows what a rule with no understanding of the failure scores on this set.
Size only. A hand-written re-implementation is usually longer than the call it replaces.
|
+16.0%+6.0% to +26.0% | 68.0% | 52.0% | 0 | $0 |
|
Leak check: a file section has no `index` line
Not ranked: shortcut baseline: shows what a rule with no understanding of the failure scores on this set.
Flags a patch when some `diff --git` section lacks git's `index` line. The edited file's section is regenerated by the same call on both sides of every pair, so the rule cannot tell them apart.
|
0.0%−7.1% to +7.1% | 100.0% | 100.0% | 0 | $0 |
How to read these numbers
- J = TPR - FPR, on the answers that carry a verdict. 0% is chance, 100% flags every bad sample and no clean one. J is the sort key within a mode; each row also has false alarms per 100 clean changes and precision at 1%, 5% and 20% prevalence.
- An answer with no verdict (failed call, blocked request, text cut by the token budget, no verdict in the text) is unreadable: it is left out of the rates and counted beside them, per label. j_strict is J under the old rule that counted such answers as 'not flagged'; j_bounds is J if every unreadable answer had been wrong, and right.
- 95% intervals. TPR and FPR: Wilson. J: Newcombe where samples are independent; where samples are grouped (task, pair, API pair, file) whole groups are resampled (100,000 draws, fixed seed).
- Best reviewer: the best ranked row of a mode that is not marked incomplete. A row with under 95% readable answers in one class is listed and ranked, with its J bounds, but is never named best reviewer or sole winner; if its J is higher than the best reviewer's it stays in the top group. Not ranked: baseline rules, shortcut baselines and rows outside a tool's design scope. Top group: the best reviewer and every ranked row whose paired difference from it includes 0. Best catalog tool: the best tool of the SAICA catalog that ran its own check; a framework that ran our rule text is not a candidate.
- A row's cost covers every call made to get its scored answers: the current answer of each sample plus earlier attempts that failed or were cut off and had to be re-run. Smoke tests, runs on earlier versions of a set and accidental duplicate answers are not included. Cost per 1,000 reviews is that cost divided by the reviews made, for one mode-specific question per review. 'Cheap' means at most $5.00 per 1,000 reviews.
- Precision is the share of flagged changes that are really bad, if the stated share of all changes were bad. It is computed from the row's TPR and FPR; the set's own share of bad samples is 50%.
The same answers, scored another way
Parts of the set, or the same set under another labelling. They show how far the headline J depends on how the set was built.
Helper file shown · 30 bad, 30 clean
Pairs that re-implement a helper of the repository. The reviewer is shown the module that defines the helper, so the hard part of real review (knowing that the helper exists) is given.
| Row | J | 95% interval | TPR | FPR |
|---|---|---|---|---|
| deepseek-v4-flash | +93.3% | +83.3% to +100.0% | 93.3% | 0.0% |
| mimo-v2.6-flash | +93.3% | +83.3% to +100.0% | 96.7% | 3.3% |
| gpt-6-luna | +90.0% | +76.7% to +100.0% | 96.7% | 6.7% |
| grok-4.1-fast | +76.7% | +60.0% to +90.0% | 96.7% | 20.0% |
| duplicate-body rule | +20.0% | +6.7% to +36.7% | 23.3% | 3.3% |
| Phi-4-mini | −25.0% | −48.3% to −1.4% | 35.7% | 60.7% |
No file shown · 20 bad, 20 clean
Pairs that re-implement a standard-library or NumPy function. The reviewer sees only the issue and the patch.
| Row | J | 95% interval | TPR | FPR |
|---|---|---|---|---|
| gpt-6-luna | +90.0% | +75.0% to +100.0% | 95.0% | 5.0% |
| deepseek-v4-flash | +75.0% | +55.0% to +90.0% | 80.0% | 5.0% |
| mimo-v2.6-flash | +75.0% | +55.0% to +90.0% | 75.0% | 0.0% |
| grok-4.1-fast | +45.0% | +25.0% to +65.0% | 90.0% | 45.0% |
| duplicate-body rule | 0.0% | −16.1% to +16.1% | 0.0% | 0.0% |
More tables for this set in results.json
top_three: J(a) - J(b) on the samples both models answered readably; 95% interval from resampled pairs. An interval that contains 0 means the two are not separated by this set.excluded_pairs: Drafted pairs left out of the set, with the reason written down at the time (handmade.py). All were left out because the clean side already re-implements something.exclusion_bound: What the 8 exclusions are worth at most: the 8 clean sides are put back as clean samples and counted as flagged; the catch rate is left as measured. A bound, not an estimate: the answers to the excluded pairs were not kept for every model.clean_sides_flagged_by_two_or_more_models: Clean sides that stayed in the set although two or more of the four models (all but Phi-4-mini) flag them. They were not re-adjudicated in this round.what_the_reviewer_seesduplicate_body_rulerounds: J per round as recorded in STATUS.md (unreadable answers counted as 'not flagged' at the time).related_work
They are under modes[].tables of results.json.
How it was tested
- 50 pairs. Each starts from one correct agent patch (Qwen3-Coder-480B in OpenHands, hidden tests pass) that uses an existing function. In the positive, that use is replaced by a hand-written re-implementation; the two patches of a pair are identical everywhere else.
- What is re-implemented: 30 of the repository's own helpers (e.g. PyJWT's base64url decoding, MONAI's ensure_tuple, oauthlib's raise_from_error, MetPy's check_units decorator), 17 standard-library functions (e.g. shlex.split, fnmatch, os.makedirs(exist_ok=True), csv.writer, functools.wraps, uuid4) and 3 NumPy functions (np.where, np.isclose, np.setdiff1d). How: 33 inline (the call replaced by a few lines), 11 rewritten helpers, 6 copies of the helper's body pasted into the patch.
- Each edit was checked: the file still parses, no undefined name is added (ruff F821), the original call is gone and the re-implementation is there. Seven hand-rolled helpers were extracted from the positive patches and run against the function they replace; they agree on the kind of input the patch passes (e.g. shlex.split 6 of 6, fnmatch 36 of 36, fountain's stringify 7 of 8: -0.0 differs). Others differ on edge cases, as hand-rolled code does (time-zone-aware indexes, error classes).
- Reviewers see the issue, the patch and, when the re-implemented function is the repository's own, the module that defines it (both patches of a pair see the same files). Same prompt for every model; temperature 0 (gpt-6-luna: its default). Run 2026-11-20. The pairs were written by Claude (the coding assistant building this benchmark), not by any reviewed model.
Caveats
- Hand-built: every re-implementation was written on purpose. Natural dependency blindness is often larger (whole helpers rebuilt) or subtler.
- sentrux, the KG's redundancy checker, was not run: its macOS release binary needs a Gatekeeper quarantine override to start, which was not approved for this run. The rule here is a simple stand-in for its documented duplicate-body check and only catches copied bodies.
- Each model asked directly was told which failure to look for and was tested on samples that have that failure or none. Its score is a single-prompt, mode-primed estimate, not a product.
- Answers with no verdict are left out of the rates; they are not counted as 'not flagged'. Unreadable: Phi-4-mini 6 of 100 (3 bad, 3 clean; mostly provider or connection errors). Marked incomplete (under 95% readable in one class): Phi-4-mini.
- The three top models are tied on this set: gpt-6-luna +90%, deepseek-v4-flash +86%, mimo-v2.6-flash +86%. gpt-6-luna - deepseek-v4-flash: +4 points, 95% interval -6 to +16; gpt-6-luna - mimo-v2.6-flash: +4 points, 95% interval -6 to +16; deepseek-v4-flash - mimo-v2.6-flash: +0 points, 95% interval -12 to +12 (50 pairs resampled). No best reviewer can be named.
- The clean side was selected on the outcome. 8 of 58 drafted pairs were dropped because the strong models flagged their clean side and a re-read found that the agent's own patch already re-implements something. Missed positives and unflagged clean sides were not re-read, so the step removed false alarms of exactly the models that are ranked. Bound: had the 8 clean sides stayed and been flagged, J would be about +77% for gpt-6-luna instead of +90%, and about +72% for deepseek-v4-flash and +72% for mimo.
- 3 clean sides that two or more models flag are still in the set (db-016, db-020, db-050); the review of 2026-11-21 reads them as the same kind of case as an excluded one, and they were not judged again here. All 50 re-implementations and every decision about what counts as clean were made in one model session (Claude); no second person or model checked them.
- For the 30 pairs that re-implement a repository helper the reviewer is shown the module that defines it. In real review, knowing that the helper exists is the hard part. On those pairs gpt-6-luna scores J +90%, deepseek-v4-flash scores J +93%, mimo-v2.6-flash scores J +93%; on the 20 pairs with no file shown +90%, +75%, +75% (same order). A variant with the file left out, or hidden among other files, has not been run.
- A rule that compares no code scores J +38%: flag the patch when no function or class of the shown file is named in the added lines (or, with no file shown, when it adds 20 or more lines). The published duplicate-body rule (+12%) finds the 6 pairs that were written as copies, plus both sides of db-051: there the shown file is changed by the patch and is shown as it is after the patch, although the prompt calls it unchanged.
- Related measurements exist and are not reproduced here: RepoReuse (Ma et al., arXiv:2609.35357) audits how often coding agents re-implement existing repository code over several turns and reports duplicated logic in 50.8% of task chains by turn 5; DevEval (arXiv:2405.19856) and EvoCodeBench (arXiv:2404.00599) score recall of reference dependencies. This mode asks a different question: does a reviewer notice a re-implementation.
- 50 pairs: every interval here is about +/-10 points or wider.
Dependency blindness on the leaderboard → How this set was built → Threats to validity → Build a supervision kit →
How to detect it
Signals that point to dependency blindness, from the SAICA knowledge graph:
- Emitted function signature matches an importable symbol in the project's declared dependencies.
- Emitted function body substantially overlaps (≥70% AST similarity) with a known stdlib symbol.
- Agent emits code despite a language-server suggestion to use an existing symbol.
- Emitted module reimplements a utility already present in the repository.
Tools in the catalog that address it
1 tool is tagged with dependency blindness in the catalog, most-starred first. A tag is the catalog's claim about what a tool is for; it is not a test result. Where SAICA-Bench ran the tool on this mode, the measured J is shown: that row is the evidence.
- sentrux Real-time architectural sensor that scores codebase modularity, acyclicity, and redundancy as the agent writes.
In other taxonomies
How dependency blindness maps to other failure taxonomies. “Exact” means the same failure, “partial” an overlap, “related” a neighbouring category.
-
9 Critical Failure Patterns of Coding Agents
Mapping drafted by an AI model from the source text; not yet reviewed by a person.
-
partial
DAP-08Codebase Awareness & Refactoring Issues — Half of DAP-08 is the definition of dependency_blindness: agents "often re-implement an algorithm from scratch, rather than using a well-established software package" (the post's example is a hand-written Levenshtein distance). DAP-08 is wider: it also covers losing track of the architecture and editing the wrong component.
Related:
DAP-07Repeated Code -
partial
Related failure modes
- Code hallucination (fabrication) — Code that calls functions, methods or packages that do not exist.
- Logic errors in AI-generated code — Real APIs used correctly, but the code computes the wrong result.
- Deprecated API usage (obsolescence) — Real APIs that are deprecated, removed or changed, often from stale training data.
Research
Questions and answers
What is dependency blindness in AI coding agents?
Dependency blindness is when an AI coding agent writes its own version of a function, class or utility that the standard library, an installed package or the codebase already provides, for example a hand-rolled argument splitter instead of shlex.split. It is about reinventing code that exists elsewhere in the project or its dependencies, not about repetition inside one generated answer. It differs from code hallucination, where the referenced code does not exist at all.
How do you detect dependency blindness?
Signals that point to it: Emitted function signature matches an importable symbol in the project's declared dependencies. Emitted function body substantially overlaps (≥70% AST similarity) with a known stdlib symbol. Agent emits code despite a language-server suggestion to use an existing symbol. Emitted module reimplements a utility already present in the repository. On SAICA-Bench: No single best reviewer: 3 rows cannot be told apart at 95% on the samples they share (gpt-6-luna, mimo-v2.6-flash and deepseek-v4-flash); the highest J is +90.0% (95% interval +82.0% to +98.0%).
Which reviewers catch dependency blindness on SAICA-Bench, and how well?
No single best reviewer: 3 rows cannot be told apart at 95% on the samples they share (gpt-6-luna, mimo-v2.6-flash and deepseek-v4-flash); the highest J is +90.0% (95% interval +82.0% to +98.0%). No tool of the catalog ran a check of its own on this set. A rule with no understanding of the failure scores J +38.0% on the same set (No name from the shown file appears in the added lines), so read the scores against that, not against 0. Each model asked directly was told which failure to look for and was tested on samples that have that failure or none. Its score is a single-prompt, mode-primed estimate, not a product.
How was detection of dependency blindness measured?
On 100 scored samples: 50 patches that re-implement something already available and 50 clean ones. Where the bad samples come from: failure planted on purpose (written by hand by Claude (the coding assistant building this benchmark): one call replaced by a re-implementation). Label: construction; each edit was checked (parses, original call gone, re-implementation present). Data: Hand-built on SWE-rebench-OpenHands-Trajectories (Nebius, CC-BY-4.0): correct agent patches, one call re-implemented by hand. J = TPR - FPR, on the answers that carry a verdict. 0% is chance, 100% flags every bad sample and no clean one. J is the sort key within a mode; each row also has false alarms per 100 clean changes and precision at 1%, 5% and 20% prevalence.
Benchmark results from 2026-11-22. The definition, detection signals and tags come from the SAICA knowledge graph.