]> dreyeck.freedombox.rocks Git - topic-maps.git/blob - docs/slice-19d2-xtm-mergemap.md
Implement noncyclic XTM mergeMap content import (Slice 22E)
[topic-maps.git] / docs / slice-19d2-xtm-mergemap.md
1 # Slice 19D2 — CTM Mergemap with XTM 2.0
2
3 Base: `f6a86668fce782a2e77d3bd763d58b0489079d75`.
4 Load **`topic-maps/ctm/xtm`** explicitly, using the already qualified 19D0B
5 source environment. Plain `topic-maps/ctm` remains usable without CXML; an XTM
6 request without the adapter signals `:xtm-mergemap-reader-required` before retrieval.
7 This configuration limit does not assert that XTM is an unknown CTM notation.
8
9 ## Normative requirements and local choices
10
11 [CTM 3.16.3](https://www.isotopicmaps.org/ctm/ctm.html) identifies the CTM and
12 XTM notations, requires immediate deserialization of the referenced resource,
13 and transfers its Topics and Associations. The Include-specific identifier
14 alias transformation from 3.16.2 does not apply to Mergemap.
15
16 The unchanged XTM reader follows [XTM 2.0 4.3–4.9 and 4.20](https://www.isotopicmaps.org/sam/sam-xtm/).
17 [TMDM 6.2, 6.6–6.7 and 7.2](https://www.isotopicmaps.org/sam/sam-model/2006-06-18/)
18 remain responsible for identity, normalization and the type-instance Core Subjects.
19 The existing [CXTM 3.18 and clause 4](https://www.isotopicmaps.org/cxtm/cxtm.html)
20 handle locator representation and canonical ordering.
21
22 Local implementation choices are an opt-in ASDF adapter, explicit caller-provided
23 resource resolution, an unchanged source-preserving `import-map-content`, and
24 conditions which retain their original causes. These choices are not claimed as
25 normative API or transaction requirements. No TMDM or XTM language rules changed.
26
27 ## Real processing path
28
29 1. CTM resolves the resource and notation IRI through its existing IRI/QName reader.
30 2. The finite caller resolver receives:
31
32 ```lisp
33 (funcall resolver absolute-resource-iri
34 :purpose :mergemap :from-document-iri parent-document-iri
35 :notation "http://psi.topicmaps.org/iso13250/xtm")
36 ;; => raw-octet-vector, diagnostic-source-key
37 ```
38
39 3. The shared resource path validates the resolver result without decoding it.
40 `mergemap-reader` selects the notation-specific function. CTM resources alone
41 use the existing UTF-8/BOM decoder; Include still uses that same CTM decoder.
42 4. The XTM adapter passes the bytes to `topic-maps.xtm:deserialize-octets`, with
43 the absolute Resource IRI as its Document IRI. Source-Key has no identity role.
44 Specialized unsigned-byte-8 vectors retain their EQ identity; accepted generic
45 byte vectors are specialized without changing their byte values.
46 5. XTM XML processing still exclusively uses `topic-maps.xml:parse-octets`.
47 6. The actual child Map is passed to `topic-maps.tmdm:import-map-content`.
48 That operation copies content and normalizes the target. The child is unchanged;
49 its root Map identifiers and root Reifier edge are not transferred.
50
51 No callback can enable implicit filesystem/network retrieval. The qualified XML
52 policy still rejects DOCTYPE and external resources. All original archives,
53 source hashes, decoder patches and Nix configuration remain unchanged.
54
55 ## Original contract and actual objects
56
57 Unmodified source `ctm/in/mergemap-xtm.ctm`:
58
59 ```ctm
60 %mergemap <mergemap-xtm.sub> http://psi.topicmaps.org/iso13250/xtm
61
62 lars isa person.
63 ```
64
65 Its Document IRI **D** is:
66 `https://ctm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/ctm/in/mergemap-xtm.ctm`.
67
68 Resource IRI and child Document IRI **R** are:
69 `https://ctm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/ctm/in/mergemap-xtm.sub`.
70
71 The resolver supplies Source-Key `ctm/in/mergemap-xtm.sub` and the unchanged
72 161 source bytes. The child reader produces five Topics, one Association and
73 two Roles. The final target has seven Topics, two Associations and four Roles:
74
75 | Actual TMDM Topic | Identifier property |
76 |---|---|
77 | CTM lars | II `D#lars` |
78 | CTM person | II `D#person` |
79 | imported XTM lars | II `R#lars` |
80 | imported XTM person | II `R#person` |
81 | Core type-instance | SI `http://psi.topicmaps.org/iso13250/model/type-instance` |
82 | Core instance | SI `http://psi.topicmaps.org/iso13250/model/instance` |
83 | Core type | SI `http://psi.topicmaps.org/iso13250/model/type` |
84
85 Both Associations have the same actual Core Type Topic. The CTM Association's
86 `instance`/`type` Players are its CTM-local Topics; the imported Association's
87 Players are its XTM-local Topics. Roles have the correct Association Parent,
88 Role Type and Player by EQ, including rolesPlayed. Locals retain one II each;
89 there are no Include aliases. Each source content object resolves to its actual
90 canonical target copy through the existing import resolver.
91
92 The generated CXTM is **byte-identical to the unchanged 1,519-byte baseline**.
93 The integration report contains both complete XML documents and observed model
94 properties. No hand-built substitute is counted as an original CTM contract.
95
96 ## Tests and diagnostic contract
97
98 The tests cover full-IRI/QName notation and resources, relative/absolute resource
99 resolution, exact resolver arguments, Source-Key independence, repeated and mixed
100 CTM/XTM directives, nested CTM-to-XTM resolution, forward Players, Core merging,
101 retained handles and mutation-free serialization. All thirteen previous positive import sources and six negative sources are also repeated after loading the adapter. Child Maps and retained raw
102 slots are checked before/after actual imports. Root Reifier exclusion is tested
103 through explicit model-API instrumentation, not claimed as new XTM syntax.
104
105 A valid UTF-16LE XML document with BOM reaches the qualified XML boundary using
106 the same octet vector by EQ. The CTM decoder is observed zero times. The bytes
107 remain unchanged; a control confirms they cannot be decoded as CTM UTF-8.
108
109 | Failure | Condition / code |
110 |---|---|
111 | no resolver | `ctm-unsupported`, `:resource-resolver-required` |
112 | resource absent or declined with NIL | `ctm-resource-error`, `:resource-unavailable` |
113 | resolver signals refusal | `ctm-resource-error`, `:resource-resolver-failure`, original cause retained |
114 | invalid octets or Source-Key result | `ctm-resource-error`, `:invalid-resource-result` |
115 | unknown notation | `ctm-rejection`, `:unknown-mergemap-notation`, before retrieval |
116 | malformed XML / invalid encoding | `mergemap-source-error`, original `xtm-xml-error` retained |
117 | XTM structure/version/namespace/ID error | `mergemap-source-error`, original `xtm-rejection` retained |
118 | valid unsupported XTM feature | `mergemap-source-unsupported`, original `xtm-unsupported` retained |
119 | model conflict during child reading | `mergemap-source-error`, original TMDM condition retained |
120 | model conflict during content import | existing `ctm-rejection`, original TMDM condition retained |
121
122 The public `topic-maps.ctm:error-cause` reader exposes that exact cause object.
123 For XML failures its nested cause remains available through the XTM reader.
124 The wrapper retains the CTM directive position and resource context; the XTM
125 condition retains its child Document IRI, Source-Key and approximate XML position.
126 Unsupported XTM features are not classified as malformed CTM notation.
127
128 ## Full offline validation
129
130 ```sh
131 cd /Users/rgb/workspace/topic-maps
132 python3 scripts/validate-slice-19d2.py \
133 --sbcl /nix/store/hlsiblxpxr1hax3dw9x8znd6zy097ddj-sbcl-2.5.4/bin/sbcl \
134 --output /tmp/topic-maps-19d2-review
135 ```
136
137 Preparation uses offline Nix evaluation and the existing hash-verified sources
138 and patches. The driver runs XML qualification (146 checks), unchanged standalone
139 XTM tests (327 checks and six original byte comparisons), the complete retained
140 suite (12,100 Lisp checks, two Python tests and 16,294 NFC comparisons), then the
141 new adapter tests in that fresh process. Network is denied with `sandbox-exec`.
142 `TOPIC_MAPS_XTM_MERGEMAP_REPORT` explicitly enables adapter tests after the core
143 suites; core reports remain separate. The restored source registry is read as
144 safe configuration data from the qualified `CL_SOURCE_REGISTRY` environment.
145
146 Only `mergemap-xtm.ctm` moves from NOT-RUN to PASS: final CTM status is
147 **205 PASS, 4 known FAIL, 64 NOT-RUN**. Original DateTime and Include discrepancies
148 remain byte-for-byte unchanged. Five obsolete XTM-unimplemented check labels in
149 the retained Mergemap/CXTM reports now describe the explicit-resolver requirement;
150 the old original-case observations and all other retained report content remain
151 unchanged. The XTM report is separate: **6 PASS, 0 FAIL, 118 NOT-RUN** across the
152 124 XTM inputs, comprising six executed positives and 118 unexecuted inputs.
153 No new XTM reader contracts are registered by the CTM import.
154
155 A full offline Nix rebuild is NOT-RUN; existing Store sources plus fresh patched
156 ASDF loading are exercised. No additional dependency downloads are needed.
157
158 ## Inspect the actual original result
159
160 First prepare and launch a fresh qualified process (the target must be new):
161
162 ```sh
163 cd /Users/rgb/workspace/topic-maps
164 xtm_registry=$(python3 scripts/prepare-cxml.py --directory /tmp/xtm-mergemap-review-sources --registry)
165 /usr/bin/sandbox-exec -p '(version 1)(allow default)(deny network*)' \
166 /usr/bin/env CL_SOURCE_REGISTRY="$xtm_registry" \
167 /nix/store/hlsiblxpxr1hax3dw9x8znd6zy097ddj-sbcl-2.5.4/bin/sbcl \
168 --no-sysinit --no-userinit
169 ```
170
171 ```lisp
172 (require :asdf)
173 (asdf:load-asd #P"/Users/rgb/workspace/topic-maps/topic-maps.asd")
174 (asdf:load-system "topic-maps/ctm/xtm")
175 (defparameter *result*
176 (topic-maps.ctm:deserialize-string
177 "%mergemap <child.xtm> <http://psi.topicmaps.org/iso13250/xtm> lars isa person."
178 :document-iri "https://example.invalid/main.ctm"
179 :resource-resolver
180 (lambda (iri &key purpose from-document-iri notation)
181 (assert (equal iri "https://example.invalid/child.xtm"))
182 (assert (eq purpose :mergemap))
183 (assert (equal from-document-iri "https://example.invalid/main.ctm"))
184 (assert (equal notation "http://psi.topicmaps.org/iso13250/xtm"))
185 (with-open-file (in #P"/Users/rgb/workspace/topic-maps/vendor/cxtm/ctm/in/mergemap-xtm.sub"
186 :element-type '(unsigned-byte 8))
187 (let ((octets (make-array (file-length in) :element-type '(unsigned-byte 8))))
188 (read-sequence octets in)
189 (values octets "pinned:ctm/in/mergemap-xtm.sub"))))))
190 (describe *result*)
191 (dolist (association (topic-maps.tmdm:topic-map-associations *result*))
192 (describe association)
193 (dolist (role (topic-maps.tmdm:association-roles association))
194 (describe (topic-maps.tmdm:association-role-player role))))
195 ```
196
197 No additional XTM features, XTM mergeMap, Include-rule changes, HyperDoc changes,
198 persistent-image loads or publication are part of this slice.