| Age | Commit message (Collapse) | Author |
|
An independent check of a rewritten batch job found differences on two new
inputs of one family (a blank numeric field the rewrite rejects on purpose
and the old program reads as zero). Each input has several output files, so
accepting the deviation meant three or four hand-written entries per input.
A top-level "approvedInputs": {"F23": "reason"} in a cases file now approves
every case that shares that input label. A reason is still required (blank
approves nothing), a case's own reason wins, a missing file is never turned
into a pass, and the result still lists the difference as approved.
|
|
Replace mean
Follow-ups from review of the tolerance check: an absolute tolerance above
1e-6 is refused like a relative one above 1%, numbers are compared as exact
decimals rather than doubles, dotted runs such as 1.2.3 or 10.0.0.1 are text
that must match exactly, and the self-check now changes the leading digit of
every sampled number to prove the tolerance cannot hide a real change.
The business-rules header says how many rules there were at extraction, since
later steps add rules. The assessment says what to do when it recommends
Rehost or Replace, which change no code and so have no build command.
|
|
The equivalence check compared bytes only, which cannot pass for numeric
code (Fortran, C, scientific and financial results) whose last digits differ
between compilers and math libraries, so those cases pushed people to mask
the numbers away. A case can now declare a tolerance, relative or absolute,
with a required reason. Only numbers written with a decimal point or an
exponent may differ within it; integers and every other byte must match, a
relative tolerance above 1% is refused, and the tolerance and the largest
difference are recorded in the result. The self-check also fails when a
tolerance is loose enough to hide a change to the leading digit of a number.
|