Skip to main content

Taxonomies

Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes

Shah, Morovati, Rahman, Khomh (Dalhousie University; Polytechnique Montréal) · varXiv v2 (2026-05-07) · first published 2026-03-06

Scope

Taxonomy of faults in agentic AI software, built from 13,602 closed issues and merged pull requests in 40 open-source repositories; 385 faults were sampled and coded by hand, and 145 practitioners rated the result. Version 2 of the paper has three lists: 34 fault types in 14 categories and four dimensions (Figure 2), 12 symptom categories (Table 1) and 12 root-cause categories (Table 2). The unit of study is a defect in the code of an agent framework, library, tool or application. It is not a failure of a coding agent on a user's task.

Source → License: arXiv preprint, CC BY 4.0 Paper →

Why it's ingested

The largest hand-coded study of faults in agentic AI software that the KG knows of (385 sampled faults, 40 repositories, 145 practitioners). It shows which SAICA failure modes have a neighbour among the defects developers report in agent software and which do not. It cannot show that the SAICA modes are complete or separate, because it studies software defects, not agent behaviour; see data/crosswalks/shah-to-saica.yml.

Categories & crosswalks

Each row is one category of this taxonomy with the SAICA failure modes mapped to it and how close each mapping is: “exact” means the same failure, “partial” an overlap, “related” a neighbouring category. 12 mappings cover 10 of the 76 categories; the 66 with no mapping are listed under them. The category ids and labels were last compared with the source on 2026-11-21.

Mapping drafted by an AI model from the source text; not yet reviewed by a person.

Categories with no SAICA failure mode

These 66 categories have no mapping. The reason is the one the crosswalk file states; where it states none, that is said.

Notes

Rebuilt on 2026-11-21 from the arXiv v2 PDF (2026-05-07): Figure 2 for the fault tree, Table 1 for symptoms, Table 2 and section 3.1.3 for root causes. All 76 labels and all counts were compared with the text of the PDF by script. The ids (SHAH-DIM, SHAH-CAT, SHAH-FT, SHAH-SYM, SHAH-RC) are SAICA-KG's; the paper publishes none. Counts are as printed: the fault tree adds up to 370, Table 1 to 352 and Table 2 to 361, while the sample is 385, and the percentages in the tables use the table totals. Version history. Version 1 of the paper (2026-03-06) had a different structure: five dimensions, 13 major categories, 37 fault categories, 13 symptom classes and 12 root causes with other names. Until 2026-11-21 this file followed v1 for the dimensions and the 13 major categories (ids SHAH-MAJ-01 to -13, now replaced by SHAH-CAT-01 to -14). Its 12 root-cause labels (for example "Incorrect Token Management", "Datetime Conversion Errors", "Exception Suppression") matched Table 2 of neither version. The ids SHAH-DIM-01 to -04 and SHAH-RC-01 to -12 are kept but now name the v2 entries, so an old reference by id must be re-read. The same rebuild was first made on 2026-08-18 (commit 4977acf) and was lost in the reset of 2026-11-19.

View raw YAML on GitHub →