Skip to main content

AI coding agents reinventing existing code (dependency blindness)

Dependency blindness is when an AI coding agent writes its own version of a function, class or utility that the standard library, an installed package or the codebase already provides, for example a hand-rolled argument splitter instead of shlex.split. It is about reinventing code that exists elsewhere in the project or its dependencies, not about repetition inside one generated answer. It differs from code hallucination, where the referenced code does not exist at all.

The code may run correctly, so its tests can pass. The knowledge graph describes it as a silent quality regression: it adds code to maintain, duplicates effort and drifts from the conventions of the ecosystem.

At a glance: On SAICA-Bench: No single best reviewer: 3 rows cannot be told apart at 95% on the samples they share (gpt-6-luna, mimo-v2.6-flash and deepseek-v4-flash); the highest J is +90.0% (95% interval +82.0% to +98.0%). Results ↓

Also called
  • reinvention
  • ecosystem duplication
  • reinventing the wheel
  • reimplementation
  • not invented here
  • missed code reuse
  • duplicate utility
  • redundant implementation
  • library underuse

How well can dependency blindness be caught?

J 80% or more on our set The agent writes its own version of something the codebase, the standard library or an installed package already provides. No single best reviewer: 3 rows cannot be told apart at 95% on the samples they share (gpt-6-luna, mimo-v2.6-flash and deepseek-v4-flash); the highest J is +90.0% (95% interval +82.0% to +98.0%).

Best reviewer

any ranked row; a tie is shown as a top group

top group of 3: gpt-6-luna, mimo-v2.6-flash and deepseek-v4-flash

Highest J +90.0% (95% interval +82.0% to +98.0%)

Best catalog tool

a tool of the catalog running its own check

No tool of the catalog ran a check of its own on this set.

Shortcut baseline

a rule that understands nothing about the failure

No name from the shown file appears in the added lines

J +38.0%. Read the reviewers against this, not against 0.

What was tested

Bad samples
Failure planted on purpose. written by hand by Claude (the coding assistant building this benchmark): one call replaced by a re-implementation. Label: construction; each edit was checked (parses, original call gone, re-implementation present). Clean samples: the same correct agent patch with the original call.
One sample
one patch; the two patches of a pair are grouped (intervals resample pairs).
Samples scored
50 patches that re-implement something already available and 50 clean (50% of the set is bad).
Measure
J 80% or more on our set.
Shortcut baseline
A rule with no understanding of the failure scores J +38.0% here (No name from the shown file appears in the added lines). Read reviewer scores against it, not against 0.
Tuning
The clean side was selected on the outcome. 8 of 58 drafted pairs were dropped because the strong models flagged their clean side and a re-read found that the agent's own patch already re-implements something. Missed positives and unflagged clean sides were not re-read, so the step removed false alarms of exactly the models that are ranked. Bound: had the 8 clean sides stayed and been flagged, J would be about +77% for gpt-6-luna instead of +90%, and about +72% for deepseek-v4-flash and +72% for mimo.

Threats to validity that apply to this set →

All results on this set

Each model asked directly was told which failure to look for and was tested on samples that have that failure or none. Its score is a single-prompt, mode-primed estimate, not a product.

Dependency blindness: ranked reviewers, best first
# Reviewer 95% interval bad samples flagged clean samples flagged No verdictanswers left out False alarmsper 100 clean changes Precisionif 1% of changes are badPrecisionif 5% of changes are badPrecisionif 20% of changes are bad per 1,000 reviews
1 gpt-6-luna, asked directly
Model asked directly · in the top group
Run card
Model
gpt-6-luna
Provider
Azure AI Foundry
What it was shown
issue text; patch (unified diff); source of the repository module(s) the patch could have reused (optional)
Counts as flagged when
flagged when the answer carries "dependency_blindness": true
Output token limit
16384
First run
2026-11-20
Runs
100; failed 0 (0.0%)
Samples
50 bad, 50 clean
Answers with no verdict
0 (0 bad, 0 clean)
Cost
$0.24 per 1,000 reviews (list); $0.02 spent on this run. List price x tokens (the provider reports no per-call charge) (Azure AI Foundry)
+90.0%+82.0% to +98.0% 96.0%86.5% to 98.9% 6.0%2.1% to 16.2% 0 6 14%46%80% $0.24list
2 mimo-v2.6-flash, asked directly
Model asked directly · in the top group
Run card
Model
mimo-v2.6-flash
Provider
Hyperbolic
What it was shown
issue text; patch (unified diff); source of the repository module(s) the patch could have reused (optional)
Counts as flagged when
flagged when the answer carries "dependency_blindness": true
Temperature
0
Output token limit
32768
First run
2026-11-20
Runs
100; failed 0 (0.0%)
Samples
50 bad, 50 clean
Answers with no verdict
0 (0 bad, 0 clean)
Cost
$0.32 per 1,000 reviews (api); $0.03 spent on this run. Charge reported by the API with each call (Hyperbolic)
+86.0%+76.0% to +94.0% 88.0%76.2% to 94.4% 2.0%0.4% to 10.5% 0 2 31%70%92% $0.32api
3 deepseek-v4-flash, asked directly
Model asked directly · in the top group
Run card
Model
deepseek-v4-flash-0731
Provider
Hyperbolic
What it was shown
issue text; patch (unified diff); source of the repository module(s) the patch could have reused (optional)
Counts as flagged when
flagged when the answer carries "dependency_blindness": true
Temperature
0
Output token limit
32768
First run
2026-11-20
Runs
100; failed 0 (0.0%)
Samples
50 bad, 50 clean
Answers with no verdict
0 (0 bad, 0 clean)
Cost
$0.41 per 1,000 reviews (api); $0.04 spent on this run. Charge reported by the API with each call (Hyperbolic)
+86.0%+76.0% to +94.0% 88.0%76.2% to 94.4% 2.0%0.4% to 10.5% 0 2 31%70%92% $0.41api
4 grok-4.1-fast (no reasoning), asked directly
Model asked directly
Run card
Model
grok-4-1-fast-non-reasoning
Provider
Azure AI Foundry
What it was shown
issue text; patch (unified diff); source of the repository module(s) the patch could have reused (optional)
Counts as flagged when
flagged when the answer carries "dependency_blindness": true
Temperature
0
Output token limit
8192
First run
2026-11-20
Runs
100; failed 0 (0.0%)
Samples
50 bad, 50 clean
Answers with no verdict
0 (0 bad, 0 clean)
Cost
$0.53 per 1,000 reviews (list); $0.05 spent on this run. List price x tokens (the provider reports no per-call charge) (Azure AI Foundry)
+64.0%+50.0% to +76.0% 94.0%83.8% to 97.9% 30.0%19.1% to 43.8% 0 30 3%14%44% $0.53list
5 Phi-4-mini, asked directly
Model asked directly
Run card
Model
Phi-4-mini-instruct
Provider
Azure AI Foundry
What it was shown
issue text; patch (unified diff); source of the repository module(s) the patch could have reused (optional)
Counts as flagged when
flagged when the answer carries "dependency_blindness": true
Temperature
0
Output token limit
1024
First run
2026-11-20
Runs
100; failed 6 (6.0%): empty 1, no verdict 2, provider error 3
Samples
50 bad, 50 clean
Answers with no verdict
6 (3 bad, 3 clean): empty 1, no verdict 2, provider error 3; under 95% of one class was readable
Cost
$0.19 per 1,000 reviews (list); $0.02 spent on this run. List price x tokens (the provider reports no per-call charge) (Azure AI Foundry)
−6.4%−24.1% to +11.4% 42.6%29.5% to 56.7% 48.9%35.3% to 62.8% 6 of 100 (incomplete) 48.9 1%4%18% $0.19list

Shown for reference, not ranked

These rows are never counted as best. A shortcut baseline shows what a rule with no understanding of the failure scores on this set; read the reviewers' scores against it.

Dependency blindness: rows shown for reference and not ranked
Row and why it is not ranked J95% interval TPR FPR No verdict Costper 1,000 reviews
Rule: a function the patch adds has the same body as one in the shown code (stand-in for sentrux)
Baseline rule
Not ranked: baseline rule: shown for reference, never counted as best.
Run card
What it was shown
patch (unified diff); source of the repository module(s) the patch could have reused
Counts as flagged when
Flagged when a function the patch adds has the same normalised body as a function in the shown repository files.
First run
2026-11-20
Runs
100; failed 0 (0.0%)
Samples
50 bad, 50 clean
Answers with no verdict
0 (0 bad, 0 clean)
Cost
free
+12.0%+4.0% to +22.0% 14.0% 2.0% 0 $0
Shortcut: no name from the shown file appears in the added lines
Shortcut baseline
Not ranked: shortcut baseline: shows what a rule with no understanding of the failure scores on this set.
Where a repository file is shown: flags the patch when none of the functions or classes that file defines is named in the lines the patch adds. Where no file is shown: flags a patch that adds 20 or more lines. It never compares code.
+38.0%+24.0% to +52.0% 56.0% 18.0% 0 $0
Shortcut: the patch adds 20 or more lines
Shortcut baseline
Not ranked: shortcut baseline: shows what a rule with no understanding of the failure scores on this set.
Size only. A hand-written re-implementation is usually longer than the call it replaces.
+16.0%+6.0% to +26.0% 68.0% 52.0% 0 $0
Leak check: a file section has no `index` line
Shortcut baseline · leak check
Not ranked: shortcut baseline: shows what a rule with no understanding of the failure scores on this set.
Flags a patch when some `diff --git` section lacks git's `index` line. The edited file's section is regenerated by the same call on both sides of every pair, so the rule cannot tell them apart.
0.0%−7.1% to +7.1% 100.0% 100.0% 0 $0
How to read these numbers
  • J = TPR - FPR, on the answers that carry a verdict. 0% is chance, 100% flags every bad sample and no clean one. J is the sort key within a mode; each row also has false alarms per 100 clean changes and precision at 1%, 5% and 20% prevalence.
  • An answer with no verdict (failed call, blocked request, text cut by the token budget, no verdict in the text) is unreadable: it is left out of the rates and counted beside them, per label. j_strict is J under the old rule that counted such answers as 'not flagged'; j_bounds is J if every unreadable answer had been wrong, and right.
  • 95% intervals. TPR and FPR: Wilson. J: Newcombe where samples are independent; where samples are grouped (task, pair, API pair, file) whole groups are resampled (100,000 draws, fixed seed).
  • Best reviewer: the best ranked row of a mode that is not marked incomplete. A row with under 95% readable answers in one class is listed and ranked, with its J bounds, but is never named best reviewer or sole winner; if its J is higher than the best reviewer's it stays in the top group. Not ranked: baseline rules, shortcut baselines and rows outside a tool's design scope. Top group: the best reviewer and every ranked row whose paired difference from it includes 0. Best catalog tool: the best tool of the SAICA catalog that ran its own check; a framework that ran our rule text is not a candidate.
  • A row's cost covers every call made to get its scored answers: the current answer of each sample plus earlier attempts that failed or were cut off and had to be re-run. Smoke tests, runs on earlier versions of a set and accidental duplicate answers are not included. Cost per 1,000 reviews is that cost divided by the reviews made, for one mode-specific question per review. 'Cheap' means at most $5.00 per 1,000 reviews.
  • Precision is the share of flagged changes that are really bad, if the stated share of all changes were bad. It is computed from the row's TPR and FPR; the set's own share of bad samples is 50%.

The same answers, scored another way

Parts of the set, or the same set under another labelling. They show how far the headline J depends on how the set was built.

Helper file shown · 30 bad, 30 clean

Pairs that re-implement a helper of the repository. The reviewer is shown the module that defines the helper, so the hard part of real review (knowing that the helper exists) is given.

RowJ95% intervalTPRFPR
deepseek-v4-flash+93.3%+83.3% to +100.0%93.3%0.0%
mimo-v2.6-flash+93.3%+83.3% to +100.0%96.7%3.3%
gpt-6-luna+90.0%+76.7% to +100.0%96.7%6.7%
grok-4.1-fast+76.7%+60.0% to +90.0%96.7%20.0%
duplicate-body rule+20.0%+6.7% to +36.7%23.3%3.3%
Phi-4-mini−25.0%−48.3% to −1.4%35.7%60.7%
No file shown · 20 bad, 20 clean

Pairs that re-implement a standard-library or NumPy function. The reviewer sees only the issue and the patch.

RowJ95% intervalTPRFPR
gpt-6-luna+90.0%+75.0% to +100.0%95.0%5.0%
deepseek-v4-flash+75.0%+55.0% to +90.0%80.0%5.0%
mimo-v2.6-flash+75.0%+55.0% to +90.0%75.0%0.0%
grok-4.1-fast+45.0%+25.0% to +65.0%90.0%45.0%
duplicate-body rule0.0%−16.1% to +16.1%0.0%0.0%
More tables for this set in results.json
  • top_three: J(a) - J(b) on the samples both models answered readably; 95% interval from resampled pairs. An interval that contains 0 means the two are not separated by this set.
  • excluded_pairs: Drafted pairs left out of the set, with the reason written down at the time (handmade.py). All were left out because the clean side already re-implements something.
  • exclusion_bound: What the 8 exclusions are worth at most: the 8 clean sides are put back as clean samples and counted as flagged; the catch rate is left as measured. A bound, not an estimate: the answers to the excluded pairs were not kept for every model.
  • clean_sides_flagged_by_two_or_more_models: Clean sides that stayed in the set although two or more of the four models (all but Phi-4-mini) flag them. They were not re-adjudicated in this round.
  • what_the_reviewer_sees
  • duplicate_body_rule
  • rounds: J per round as recorded in STATUS.md (unreadable answers counted as 'not flagged' at the time).
  • related_work

They are under modes[].tables of results.json.

How it was tested

  • 50 pairs. Each starts from one correct agent patch (Qwen3-Coder-480B in OpenHands, hidden tests pass) that uses an existing function. In the positive, that use is replaced by a hand-written re-implementation; the two patches of a pair are identical everywhere else.
  • What is re-implemented: 30 of the repository's own helpers (e.g. PyJWT's base64url decoding, MONAI's ensure_tuple, oauthlib's raise_from_error, MetPy's check_units decorator), 17 standard-library functions (e.g. shlex.split, fnmatch, os.makedirs(exist_ok=True), csv.writer, functools.wraps, uuid4) and 3 NumPy functions (np.where, np.isclose, np.setdiff1d). How: 33 inline (the call replaced by a few lines), 11 rewritten helpers, 6 copies of the helper's body pasted into the patch.
  • Each edit was checked: the file still parses, no undefined name is added (ruff F821), the original call is gone and the re-implementation is there. Seven hand-rolled helpers were extracted from the positive patches and run against the function they replace; they agree on the kind of input the patch passes (e.g. shlex.split 6 of 6, fnmatch 36 of 36, fountain's stringify 7 of 8: -0.0 differs). Others differ on edge cases, as hand-rolled code does (time-zone-aware indexes, error classes).
  • Reviewers see the issue, the patch and, when the re-implemented function is the repository's own, the module that defines it (both patches of a pair see the same files). Same prompt for every model; temperature 0 (gpt-6-luna: its default). Run 2026-11-20. The pairs were written by Claude (the coding assistant building this benchmark), not by any reviewed model.

Caveats

  • Hand-built: every re-implementation was written on purpose. Natural dependency blindness is often larger (whole helpers rebuilt) or subtler.
  • sentrux, the KG's redundancy checker, was not run: its macOS release binary needs a Gatekeeper quarantine override to start, which was not approved for this run. The rule here is a simple stand-in for its documented duplicate-body check and only catches copied bodies.
  • Each model asked directly was told which failure to look for and was tested on samples that have that failure or none. Its score is a single-prompt, mode-primed estimate, not a product.
  • Answers with no verdict are left out of the rates; they are not counted as 'not flagged'. Unreadable: Phi-4-mini 6 of 100 (3 bad, 3 clean; mostly provider or connection errors). Marked incomplete (under 95% readable in one class): Phi-4-mini.
  • The three top models are tied on this set: gpt-6-luna +90%, deepseek-v4-flash +86%, mimo-v2.6-flash +86%. gpt-6-luna - deepseek-v4-flash: +4 points, 95% interval -6 to +16; gpt-6-luna - mimo-v2.6-flash: +4 points, 95% interval -6 to +16; deepseek-v4-flash - mimo-v2.6-flash: +0 points, 95% interval -12 to +12 (50 pairs resampled). No best reviewer can be named.
  • The clean side was selected on the outcome. 8 of 58 drafted pairs were dropped because the strong models flagged their clean side and a re-read found that the agent's own patch already re-implements something. Missed positives and unflagged clean sides were not re-read, so the step removed false alarms of exactly the models that are ranked. Bound: had the 8 clean sides stayed and been flagged, J would be about +77% for gpt-6-luna instead of +90%, and about +72% for deepseek-v4-flash and +72% for mimo.
  • 3 clean sides that two or more models flag are still in the set (db-016, db-020, db-050); the review of 2026-11-21 reads them as the same kind of case as an excluded one, and they were not judged again here. All 50 re-implementations and every decision about what counts as clean were made in one model session (Claude); no second person or model checked them.
  • For the 30 pairs that re-implement a repository helper the reviewer is shown the module that defines it. In real review, knowing that the helper exists is the hard part. On those pairs gpt-6-luna scores J +90%, deepseek-v4-flash scores J +93%, mimo-v2.6-flash scores J +93%; on the 20 pairs with no file shown +90%, +75%, +75% (same order). A variant with the file left out, or hidden among other files, has not been run.
  • A rule that compares no code scores J +38%: flag the patch when no function or class of the shown file is named in the added lines (or, with no file shown, when it adds 20 or more lines). The published duplicate-body rule (+12%) finds the 6 pairs that were written as copies, plus both sides of db-051: there the shown file is changed by the patch and is shown as it is after the patch, although the prompt calls it unchanged.
  • Related measurements exist and are not reproduced here: RepoReuse (Ma et al., arXiv:2609.35357) audits how often coding agents re-implement existing repository code over several turns and reports duplicated logic in 50.8% of task chains by turn 5; DevEval (arXiv:2405.19856) and EvoCodeBench (arXiv:2404.00599) score recall of reference dependencies. This mode asks a different question: does a reviewer notice a re-implementation.
  • 50 pairs: every interval here is about +/-10 points or wider.

Dependency blindness on the leaderboard → How this set was built → Threats to validity → Build a supervision kit →

How to detect it

Signals that point to dependency blindness, from the SAICA knowledge graph:

  • Emitted function signature matches an importable symbol in the project's declared dependencies.
  • Emitted function body substantially overlaps (≥70% AST similarity) with a known stdlib symbol.
  • Agent emits code despite a language-server suggestion to use an existing symbol.
  • Emitted module reimplements a utility already present in the repository.

Tools in the catalog that address it

1 tool is tagged with dependency blindness in the catalog, most-starred first. A tag is the catalog's claim about what a tool is for; it is not a test result. Where SAICA-Bench ran the tool on this mode, the measured J is shown: that row is the evidence.

  • sentrux Real-time architectural sensor that scores codebase modularity, acyclicity, and redundancy as the agent writes.

In other taxonomies

How dependency blindness maps to other failure taxonomies. “Exact” means the same failure, “partial” an overlap, “related” a neighbouring category.

  • 9 Critical Failure Patterns of Coding Agents

    Mapping drafted by an AI model from the source text; not yet reviewed by a person.

    • partial DAP-08 Codebase Awareness & Refactoring Issues — Half of DAP-08 is the definition of dependency_blindness: agents "often re-implement an algorithm from scratch, rather than using a well-established software package" (the post's example is a hand-written Levenshtein distance). DAP-08 is wider: it also covers losing track of the architecture and editing the wrong component.

    Related: DAP-07 Repeated Code

All 11 AI coding agent failure modes → · Glossary

Research

Questions and answers

What is dependency blindness in AI coding agents?

Dependency blindness is when an AI coding agent writes its own version of a function, class or utility that the standard library, an installed package or the codebase already provides, for example a hand-rolled argument splitter instead of shlex.split. It is about reinventing code that exists elsewhere in the project or its dependencies, not about repetition inside one generated answer. It differs from code hallucination, where the referenced code does not exist at all.

How do you detect dependency blindness?

Signals that point to it: Emitted function signature matches an importable symbol in the project's declared dependencies. Emitted function body substantially overlaps (≥70% AST similarity) with a known stdlib symbol. Agent emits code despite a language-server suggestion to use an existing symbol. Emitted module reimplements a utility already present in the repository. On SAICA-Bench: No single best reviewer: 3 rows cannot be told apart at 95% on the samples they share (gpt-6-luna, mimo-v2.6-flash and deepseek-v4-flash); the highest J is +90.0% (95% interval +82.0% to +98.0%).

Which reviewers catch dependency blindness on SAICA-Bench, and how well?

No single best reviewer: 3 rows cannot be told apart at 95% on the samples they share (gpt-6-luna, mimo-v2.6-flash and deepseek-v4-flash); the highest J is +90.0% (95% interval +82.0% to +98.0%). No tool of the catalog ran a check of its own on this set. A rule with no understanding of the failure scores J +38.0% on the same set (No name from the shown file appears in the added lines), so read the scores against that, not against 0. Each model asked directly was told which failure to look for and was tested on samples that have that failure or none. Its score is a single-prompt, mode-primed estimate, not a product.

How was detection of dependency blindness measured?

On 100 scored samples: 50 patches that re-implement something already available and 50 clean ones. Where the bad samples come from: failure planted on purpose (written by hand by Claude (the coding assistant building this benchmark): one call replaced by a re-implementation). Label: construction; each edit was checked (parses, original call gone, re-implementation present). Data: Hand-built on SWE-rebench-OpenHands-Trajectories (Nebius, CC-BY-4.0): correct agent patches, one call re-implemented by hand. J = TPR - FPR, on the answers that carry a verdict. 0% is chance, 100% flags every bad sample and no clean one. J is the sort key within a mode; each row also has false alarms per 100 clean changes and precision at 1%, 5% and 20% prevalence.

Benchmark results from 2026-11-22. The definition, detection signals and tags come from the SAICA knowledge graph.

View raw YAML on GitHub →