]> dreyeck.freedombox.rocks Git - topic-maps.git/blob - docs/slice-20a-registration.md
perf(tmdm): index association normalization merge candidates
[topic-maps.git] / docs / slice-20a-registration.md
1 # Slice 20A: register forty existing CTM original contracts
2
3 This slice registers exactly the forty positive originals authorized for Slice
4 20A. It adds no reader, model, XML, XTM or canonicalization semantics. Work resumed
5 on `codex/ctm-slice-20a`, parent
6 `9c196ddc16cf5cd727fcde81ad7835edefa0f787`, with the existing ASDF addition and
7 153-line test draft preserved and completed. Their observed starting SHA-256
8 digests are recorded in the new execution evidence.
9
10 The previous Slice 19D2 result is **205 PASS, 4 FAIL, 64 NOT-RUN**. The expected
11 result after forty successful new comparisons is **245 PASS, 4 FAIL, 24 NOT-RUN**.
12 The actual result and every candidate's status are recorded in
13 [execution evidence](../reports/slice-20a/execution.json) and the
14 [forty-contract table](../reports/slice-20a/contracts.md). Prior exploratory
15 matches are not counted as executed contracts. No separate complete historical
16 Slice-19 exploration inventory was found in the repository; the approved explicit
17 forty-case list determines this selection.
18
19 ## Registration and evidence
20
21 `tests/ctm-registered-originals.lisp` selects the existing discovered Case Records:
22 10 comments/empty Map, 13 identities/IRIs/QNames, 9 occurrences, 4 prefixes and 4
23 escapes. It checks source/baseline identity and the existing pinned Document IRI
24 before deserializing each original file through the public CTM API. It verifies
25 concrete TMDM properties and EQ references, canonicalizer purity, deterministic
26 octets, original source preservation and unchanged contract identities. Only an
27 actual byte comparison can produce `cxtm-match`; errors and unsupported features
28 retain their own outcomes. No failure becomes an empty Map or a successful test.
29
30 The new test runs permanently at the end of `scripts/fresh-sbcl.lisp`, after the
31 retained suites and optional XTM adapter. Its ASDF system can also run directly;
32 without XTM integration, only the already loaded core contracts contribute to
33 its entry totals. It does not reset or rediscover the shared corpus. All forty
34 selected cases must enter as NOT-RUN. Nonselected records and all earlier CXTM
35 observations must remain unchanged.
36
37 Historical reports keep their original bytes. The existing 19D2 validator is
38 reused without modification; it validates XML, standalone XTM, all retained
39 suites and XTM Mergemap, writing its stage snapshots before registration. The
40 new validator independently compares all forty emitted outputs against the
41 pinned baseline bytes and verifies that only those forty records changed.
42 The four known FAILs remain the same actual CXTM mismatches:
43 `include-template`, `include-wildcard2`, `occurrence-datetime2` and
44 `occurrence-datetime-timezone2`. Their complete old case records and actual
45 outputs must remain unchanged. The separate XTM corpus remains **6 PASS,
46 0 FAIL, 118 NOT-RUN** (107 positive and 11 negative unexecuted inputs).
47
48 ## Reproduce the full offline validation
49
50 Use a new output directory; existing evidence is never overwritten:
51
52 ```sh
53 cd /Users/rgb/workspace/topic-maps
54 python3 scripts/validate-slice-20a.py \
55 --sbcl /nix/store/hlsiblxpxr1hax3dw9x8znd6zy097ddj-sbcl-2.5.4/bin/sbcl \
56 --output /tmp/topic-maps-20a-review
57 ```
58
59 All Lisp execution uses fresh SBCL 2.5.4 with network denied by `sandbox-exec`.
60 Source preparation reuses hash-verified Store sources and the existing XML
61 patches, using offline Nix evaluation. No dependencies are downloaded. Retained
62 checks are 12,100 Lisp checks, 146 XML qualification checks, 327 standalone XTM
63 checks, 217 XTM-Mergemap checks, two Python unit tests and 16,294 NFC comparisons.
64 The new registration assertions and all actual byte counts are recorded in the
65 new reports. A complete Nix rebuild remains NOT-RUN.
66
67 Evidence files:
68
69 - `reports/slice-20a/current.validation.json`: current Case Records and totals.
70 - `reports/slice-20a/current.cxtm.json`: all old and forty new actual comparisons.
71 - `reports/slice-20a/current.registration.json`: model observations and checks.
72 - `reports/slice-20a/candidates.json`: per-case source, baseline and actual SHA-256.
73 - `reports/slice-20a/actual/`: forty actual CXTM byte streams.
74 - `reports/slice-20a/retained/`: full fresh qualification and regression evidence.
75 - `reports/slice-20a/integrity.json`: unchanged protected and historical file hashes.
76
77 ## Remaining cases
78
79 The 24 remaining cases are **NOT-RUN**, not failures or inferred unsupported
80 results. Their original records are listed in
81 [remaining.json](../reports/slice-20a/remaining.json). They were not in the
82 approved forty-positive registration selection; no language or negative-contract
83 extension is authorized in this slice.
84
85 | Unexecuted family | Cases | Reason to retain NOT-RUN in 20A |
86 | --- | ---: | --- |
87 | `empty2` | 1 positive | Version directive contract is outside the selected empty-map case. |
88 | `encoding-8859-5` | 1 positive | Legacy encoding contract; further encoding support is excluded. |
89 | `occurrence-ctm-integer` | 1 positive | CTM integer `*` value contract; additional literal forms are excluded. |
90 | `occurrence-string-triple`, `occurrence-string-triple2`, `unicode-escape-u4-triple-string`, `unicode-escape-u6-triple-string` | 4 positive | Triple-string contracts are outside the selected escapes and strings. |
91 | `template-ambiguous-parameter5`, `template-ambiguous-parameter6`, `template-subject-identifier2` | 3 positive | Contextual template argument contracts remain outside this registration slice. |
92 | `BOM-contradicts-encoding` | 1 negative | Encoding rejection contract is unregistered and unexecuted. |
93 | `occurrence-double-negative-infinity`, `occurrence-double-positive-infinity` | 2 negative | Infinity literal rejection contracts are unregistered and unexecuted. |
94 | `qname-hyphen`, `topic-id-illegal` | 2 negative | Identifier boundary rejection contracts are unregistered and unexecuted. |
95 | `string-unbalanced`, `string-unicode`, `string-unicode2` | 3 negative | String/escape rejection contracts are unregistered and unexecuted. |
96 | `template-item-identifier-2nd-arg`, `template-item-identifier3`, `template-subject-locator-2nd-arg`, `template-subject-locator3` | 4 negative | Template argument rejection contracts are unregistered and unexecuted. |
97 | `unknown-directive`, `wrong-version` | 2 negative | Directive/version rejection contracts are unregistered and unexecuted. |
98
99 ## Inspect selected new comparisons
100
101 These forms run from the repository in a fresh SBCL process. They load only the
102 existing foundation, CTM and CXTM systems and expose actual model objects. The
103 validation driver executes these same forms with network denied.
104
105 ```lisp
106 (require :asdf)
107 (asdf:initialize-source-registry '(:source-registry :ignore-inherited-configuration))
108 (asdf:load-asd (merge-pathnames "topic-maps.asd" (truename ".")))
109 (asdf:load-system "topic-maps")
110 (asdf:load-system "topic-maps/ctm")
111 (asdf:load-system "topic-maps/cxtm")
112 (defun inspect-original (name)
113 (let* ((corpus (topic-maps.test-foundation:discover-corpus))
114 (case (first (topic-maps.test-foundation:select-cases
115 corpus :id (format nil "ctm/in/~A.ctm" name))))
116 (doc (topic-maps.test-foundation:test-case-document-iri case))
117 (map (topic-maps.ctm:deserialize-file
118 (topic-maps.test-foundation:test-case-source-pathname case)
119 :document-iri doc))
120 (actual (topic-maps.cxtm:canonicalize-octets map :base-locator doc)))
121 (with-open-file (in (topic-maps.test-foundation:test-case-expected-baseline case)
122 :element-type '(unsigned-byte 8))
123 (let ((expected (make-array (file-length in) :element-type '(unsigned-byte 8))))
124 (read-sequence expected in)
125 (assert (equalp actual expected))))
126 (format t "~&~A: ~D byte-equal CXTM bytes~%" name (length actual))
127 (describe map)
128 (dolist (topic (topic-maps.tmdm:topic-map-topics map)) (describe topic))
129 map))
130 (inspect-original "comment-ml-nested2")
131 (inspect-original "topic-identifier-unicode")
132 (inspect-original "occurrence-string-multiline2")
133 ```
134
135 No HyperDoc changes, persistent Lisp image changes, push, deployment or next
136 slice are part of this work.