]> dreyeck.freedombox.rocks Git - topic-maps.git/blob - docs/slice-20c-prolog.md
Document Topic Maps conformance closure plan for Slice 22A
[topic-maps.git] / docs / slice-20c-prolog.md
1 # Slice 20C: CTM prolog and three negative original contracts
2
3 Parent: `d024dc96ce091db06b9d09ea7c1ccb5f90b2886d`.
4 Branch: `codex/ctm-slice-20c`. The initial index and worktree were clean; the
5 observed registered CTM state was **252 PASS, 4 FAIL, 17 NOT-RUN**.
6
7 ## Normative cause review
8
9 Reference: [ISO/IEC 13250-6:2010 CTM text](https://www.isotopicmaps.org/ctm/ctm.html),
10 sections 3.2, 3.5, 3.6, 3.7 and 3.16, consulted 2026-10-19. Published ISO
11 standards remain normative; the pinned corpus targets the 2010-03-31 draft.
12
13 Encoding selection precedes decoding: supplied encoding, UTF-8 BOM, declaration,
14 then the UTF-8 default. Conflicting explicit selections are errors. The encoding
15 header belongs at the stream start; the optional version follows it, before
16 ordinary directives or body content. Only version `1.0` is valid. `%ctm` is not
17 one of the grammar's directive types.
18
19 | Original source | Actual original bytes / directive | Required cause |
20 | --- | --- | --- |
21 | `BOM-contradicts-encoding.ctm` | `EF BB BF` followed by `%encoding "iso-8859-1"`, LF LF, `topic.`, LF | UTF-8 BOM conflicts with the explicit declaration, before decoding. |
22 | `unknown-directive.ctm` | `%ctm 1.0` and LF | Unknown directive type; syntax rejection. |
23 | `wrong-version.ctm` | `%version 10.0` and LF | Recognized version directive with a prohibited version. |
24
25 All three previously raised `ctm-unsupported / unsupported-directive` at character
26 zero. That fresh-process preimplementation observation is preserved in
27 `reports/slice-20c/preimplementation.json` for the original implementation run.
28 Actual final conditions, byte/character offsets, line/column values and complete
29 original hex strings are in `current.prolog.json` and `candidates.json`.
30 Byte offsets count from the original input, including the BOM; byte errors have
31 no invented character line/column. Character positions exclude a removed BOM.
32
33 ## Scope assessment and targeted implementation
34
35 A general encoding architecture is unnecessary. `decode-ctm-octets` checks a
36 small initial ASCII header and calls the existing unchanged strict UTF-8 decoder.
37 Only the four CTM files `unicode.lisp`, `lexer.lisp`, `reader.lisp` and
38 `resources.lisp` need implementation changes. No new dependencies are added.
39
40 The public file API adds the optional `:encoding` string, following the same
41 selection order. Matching registered UTF-8 names are compared case-insensitively;
42 there is no alias registry. Other charsets remain `ctm-unsupported /
43 unsupported-encoding` when no conflicting declaration exists. ISO-8859-5 is not
44 decoded. Supplied encoding metadata is per input stream, not inherited by
45 separate Include/Mergemap resources. Their existing resolver still returns only
46 untouched octets and the optional diagnostic source key.
47
48 The shared resource decoder uses the same preflight. XTM raw bytes continue
49 through the unchanged XTM adapter and qualified XML boundary. Unicode decoding,
50 BOM removal, malformed UTF-8 rejection, document identity, wildcard state,
51 Include aliases/templates and Mergemap content import retain their existing
52 behavior.
53
54 `read-prolog` handles the initial encoding header and exact version spelling
55 before ordinary directives. Misplaced or repeated encoding/version directives
56 receive distinct placement errors; invalid versions do not enter the numeric
57 Occurrence lexer. Unknown directives become `ctm-rejection / unknown-directive`.
58 Known invalid syntax, unsupported charsets/literals and explicit resolver policy
59 retain separate classifications.
60
61 ## Durable tests and registration
62
63 `tests/ctm-prolog.lisp` reuses the existing Case Records, result recording,
64 condition and model helpers. ASDF system `topic-maps/ctm/prolog-tests` runs after
65 the unchanged Slice 20A/20B snapshots in `scripts/fresh-sbcl.lisp`.
66
67 Positive controls cover empty and nonempty version prologs, an immediately
68 adjacent Topic after the fixed `1.0` terminal, comments, encoding
69 before version, prefixes and map reification, plus all 18 combinations of
70 optional supplied UTF-8, BOM and UTF-8 declaration. Actual Unicode Name values
71 and byte-exact canonical models are compared. Both Include and CTM Mergemap
72 exercise BOM, declared UTF-8, version and multibyte content through real resource
73 calls; input bytes and actual object references are checked.
74
75 Negative controls distinguish encoding conflicts, unsupported charsets, invalid
76 UTF-8, unknown directives, known malformed directives, unsupported literals,
77 invalid version spellings and misplaced/repeated prologs. Test instrumentation
78 observes zero UTF-8 decoder calls for conflict and unsupported-charset preflight,
79 even with a malformed body. Resource failures retain the child source and
80 Include/Mergemap context. The UTF-8 decoder is always restored after observation.
81
82 Each original is read twice through the public file API with its existing
83 Document IRI. Exact condition/code/position/source, reproducibility, unchanged
84 source bytes, unchanged contract identity and an accepted file control are
85 required before `record-result` registers PASS. Only the three selected records
86 change. In particular `empty2`, the valid ISO-8859-5 input and all other open
87 literal/template cases remain NOT-RUN.
88
89 One obsolete historical assertion that `%version 1.0` is unsupported is replaced
90 by a positive empty-Map assertion with the same assertion count. Its current
91 label changes in the prefix report and the combined core CXTM report. Historical
92 report files keep their exact bytes; current comparisons allow only that label
93 change and the five previously documented 19D2 label changes. Prior original
94 observations and the four known DateTime/Include FAILs remain unchanged.
95
96 ## Full offline validation and evidence
97
98 ```sh
99 cd /Users/rgb/workspace/topic-maps
100 python3 scripts/validate-slice-20c.py \
101 --sbcl /nix/store/hlsiblxpxr1hax3dw9x8znd6zy097ddj-sbcl-2.5.4/bin/sbcl \
102 --output /tmp/topic-maps-20c-review
103 ```
104
105 Use a new output directory. The 20C driver reuses the existing offline source
106 preparation, qualification programs, `scripts/test.sh`, ASDF suites and report
107 format. Older 20A/20B validators intentionally prohibit reader changes, so the
108 20C driver runs their same suites and verifies the retained stage reports rather
109 than weakening those historical validators. All Lisp qualification, regression
110 and documented retained inspections run in fresh SBCL 2.5.4 with network denied.
111 Nix preparation is offline and hash-verifies existing Store sources and patches.
112 No dependency downloads or full Nix rebuild occur.
113
114 Expected CTM counts after three proved rejections are **255 PASS, 4 FAIL,
115 14 NOT-RUN**; [execution.json](../reports/slice-20c/execution.json) records actual
116 validated totals. Separate XTM remains **6 PASS, 0 FAIL, 118 NOT-RUN**.
117
118 - `current.prolog.json`: original conditions, positive models and negative controls.
119 - `current.validation.json`: final original Case Records and counts.
120 - `current.cxtm.json`: all retained positive comparisons and four known mismatches.
121 - `candidates.json`: original source SHA-256 values and exact diagnostics.
122 - `retained/`: core, XTM-Mergemap, Slice 20A and Slice 20B stage reports.
123 - `xml.log`, `xtm.log`, `regression.log`: complete actual execution output.
124 - `source-preparation.json`, `integrity.json`: verified dependency provenance,
125 protected files and unchanged historical reports.
126
127 Implementation, tests and execution evidence are committed together only after
128 successful full validation. No HyperDoc changes, push, deployment or later slice
129 are included.