]> dreyeck.freedombox.rocks Git - topic-maps.git/blob - docs/slice-19c-validation.md
docs: design draft for a read-only SHOP3 adapter over TMDM objects
[topic-maps.git] / docs / slice-19c-validation.md
1 # Slice 19C validation and evidence
2
3 Base: `5e716c078b885e97ebe858230ce7d8578e65ae4b`.
4 Branch: `codex/ctm-slice-19c`.
5 Corpus pin: `cd165b150176c1d6842b37be9cc9051b40d4996b`.
6
7 ```sh
8 cd /Users/rgb/workspace/topic-maps
9 PYTHONDONTWRITEBYTECODE=1 python3 scripts/validate-slice-19c.py \
10 --sbcl /nix/store/hlsiblxpxr1hax3dw9x8znd6zy097ddj-sbcl-2.5.4/bin/sbcl
11 ```
12
13 The validator executes all retained suites and the Include suite in one fresh
14 SBCL process under macOS `sandbox-exec` with network access denied. The normal
15 source registry/output settings remain unchanged. No additional runtime,
16 relocation, cache redesign, persistent image or HyperDoc system is involved.
17 The existing Python infrastructure tests and independent NFC oracle also run.
18
19 `reports/slice-19c/execution.json` records exact commands, UTC times, runtime,
20 phase totals, status changes and source/artifact hashes. `tests.log` is the
21 actual full execution. All previous phase totals and every pre-existing corpus
22 case must remain unchanged. Original positive outputs and observed model graphs
23 are compared with Slice 19B. Four old unsupported-Include labels are replaced
24 with precise missing-resolver assertions, without changing phase counts.
25 Original Template error classes/codes/sources/positions remain unchanged;
26 Template context strings gain definition/invocation source metadata.
27
28 All twelve original Include contracts now execute. Seven positive comparisons
29 match unmodified bytes and three negatives reject for their intended reasons.
30 `include-template.ctm` and `include-wildcard2.ctm` have their real comparison
31 results registered as **FAIL**, separately from passing semantic assertions.
32 See `reading-ctm-include.md` for the derivation and exact identifier differences.
33 The expected aggregate is **204 PASS, 4 FAIL, 0 UNSUPPORTED, 65 NOT-RUN**,
34 including the two retained DateTime FAILs. This is not an all-green conformance
35 claim. A successful regression means those discrepancies remain visible and no
36 unexpected result occurs.
37
38 `include.json` contains the original sources, actual model properties, resource
39 requests, every successful processor's Document/Wildcard IRI and Counter, and
40 Topic identifiers before/after each actual alias pass. The fresh test process
41 instruments the real functions, delegates to them, captures observations, and
42 restores the functions afterward. It supplies no alternative import or identity
43 model. `check-include.py` independently checks these traces and model identities.
44 `cxtm.json` plus `actual/` preserve all nine actual canonical results, including
45 the two mismatches. `check-cxtm.py` compares original expected bytes independently,
46 checks canonical XML and reference integrity, and never converts mismatch to PASS.
47
48 Focused assertions cover preserved identifier categories/values, literal-prefix
49 aliasing and percent handling, resolver identity, counter/cache locality,
50 Template arguments/forward references/overloads, correct nested source positions,
51 canonical handles for six content kinds, root metadata exclusion, Scope/Variant/
52 Reifier integrity, pure serialization and rejection before caller Map/Counter/
53 registry mutation. Resource errors and cycle/depth policy remain distinct from
54 malformed CTM. Invalid XTM-as-Include is rejected by the CTM reader at the child
55 source, not by an unrelated I/O failure or later undefined prefix.
56
57 The documented inspection runs separately in a fresh offline process. A base
58 lexer counterprobe reloads the committed pre-Include lexer in a disposable
59 process and confirms its former unsupported-directive response. Corpus/reference
60 hashes and TMDM/CXTM source trees must remain unchanged. No original baseline is
61 rewritten and no XTM-deserialization contract is promoted.