summaryrefslogtreecommitdiff
path: root/plugins/code-modernization/scripts/tests/test_proof_tools.py
AgeCommit message (Collapse)Author
2026-09-25code-modernization: a rule is tested only if a test that ran backs itMorgan Lunt
A rule named only by a skipped or pending test no longer counts as covered. It is listed as "named, not run" and fails the rules check. A test-only folder is listed as tooling and gets no verdict. The verify command tries malformed records only when the person asked for exact behavior, so a proof loop has an end. The report parses rule headings and fields in linear time, and the report and pane show the new status.
2026-09-24code-modernization: an uplift is not proven by a typed baseline or edited testsMorgan Lunt
Three ways a same-technology uplift could come out PROVEN too easily: - The baseline came from a table someone typed. It now counts as measured only when the script can read per-test results (JUnit or TRX XML, a runner log with a summary line, or per-test JSON) from analysis/<system>/baseline/ or a "Recorded:" line of BASELINE.md; a counts-only file is ignored, two measured sources that disagree are a gap, and the typed table must match what was measured. - Tests edited during the uplift passed unnoticed. The pack now compares the legacy and working-copy test files (by walking files only) and lists what was added, removed or changed; a removed file, more than a quarter of the files changed, or no legacy tests to compare with is a gap, and it says plainly that weakened assertions cannot be detected. - A silent behavioral change with no test at its site. Each Behavioral-silent delta in the catalog must have a test file naming its site file, or the module is at most PARTLY PROVEN. The report's Proof section shows the three new checks, and an uplift pack that lacks any of them reads as PARTLY PROVEN. 214 script tests pass.
2026-09-24code-modernization: an independent proof step with one verdict per moduleMorgan Lunt
/code-modernization:modernize-verify re-checks a finished build instead of trusting the notes it left behind, and a script (not the model) computes one verdict per module: PROVEN, PARTLY PROVEN or NOT PROVEN, with the rules written into the output. The pack re-reads the tests' own result files or a saved runner log (counts the command merely reported never count), traces every P0 business rule to a test that names it, judges the old and new outputs again with compare.py, then runs at least ten inputs that development did not use on both systems. It requires a recorded canary that turned tests red, checks that the legacy source was not touched (by file times: no process runs in the analyzed tree), and lists every human criterion as waiting for a person. For an uplift it diffs the fresh results against the recorded baseline. Verdicts are never carried over from an earlier run. The report gains a Proof section that recomputes each verdict from the same checks, and a failed diagram no longer stops the ones after it from drawing.