pytest
14,730Python test framework — discipline gate for "done means tests pass."
Activity band details
Activity band A to F: a bin of a score that adds GitHub stars (log scale), a push to the upstream repository in the 12 months before `as_of`, the number of declared integration surfaces and, where the catalogue records them, citing papers and benchmark evaluations. It measures popularity and activity. It is not a level of evidence: no band means a paper cites the tool or that it was measured. Recency is judged as of 2026-11-22.
Computed band: C
(raw score 5.17).
Shown band: C
.
Facets:
documented_in=0evaluated_on=0integration_surfaces=1.5paper_citations=0recent_push=2stars=1.67
Source of truth: pipeline/maturity/score.py
Description
pytest is the de facto Python test framework: it discovers and runs unit, integration, and functional tests, reporting failures in human-readable form. It is the usual gate for the "done means tests pass" rule: failing assertions show logic errors that only appear when code runs. Its summary also counts skipped and xfailed tests, which a person can read for unfinished work or weakened tests; pytest cannot tell a legitimate skip from a manipulated test. Configuration lives in pyproject.toml [tool.pytest.ini_options], which the audit detector now recognises.
Why it's included
Occupies detection × post_generation × fully_autonomous for logic_error in the Python lane. For incomplete_execution and test_manipulation it only records evidence (skip and xfail counts); the 2026-11-20 tag audit moved both to records_failure_modes (research/fm_tag_audit/final_table.csv). Mature workhorse; missing it leaves the audit detector unable to recognise the single most common testing configuration in Python repos.
Security notes
Standard dev-tool security posture; pytest executes user-authored test code with full developer privileges, so test files should be reviewed with the same scrutiny as production code. Plugin ecosystem is large and worth pinning by version.
Addresses failure modes
These tags are the catalog's claim about what pytest is built to catch or prevent. A tag is not a test result. SAICA-Bench has not measured pytest, so none of these tags is tested here.
- Logic error — Correct API use, incorrect logic.
Records evidence of
pytest can record evidence of these failures (traces, or checks you configure yourself), but it does not detect or prevent them on its own. They are not counted as failure modes it addresses.