]> dreyeck.freedombox.rocks Git - topic-maps.git/blob - docs/slice-19d1-xtm-identity.md
Implement noncyclic XTM mergeMap content import (Slice 22E)
[topic-maps.git] / docs / slice-19d1-xtm-identity.md
1 # Slice 19D1 — Reading XTM 2.0 identity
2
3 Base: `1691c8e0712e30666de2e667a5050eccb0086f71`. This is an independent
4 `topic-maps/xtm` ASDF system depending on `topic-maps/tmdm` and `topic-maps/xml`.
5 It neither depends on CTM nor changes the existing information-item model.
6
7 ## Normative basis and implementation boundary
8
9 The [published-equivalent XTM 2.0 text](https://www.isotopicmaps.org/sam/sam-xtm/)
10 is ISO/IEC 13250-3:2007, not XTM 2.1. Clauses 4.1–4.2 establish the XML Infoset
11 input; 4.3.1 and Annex A specify namespaces and grammar. Clauses 4.3.3–4.3.4
12 require percent-UTF-8 decoding and Document-IRI resolution. Clauses 4.4–4.8
13 specify map creation, required topic IDs and the three identity properties.
14 Clause 4.9 maps `instanceOf` to type-instance Associations. Clause 4.20 searches
15 existing SI/II locators before creating a reference Topic; a hit adds no II.
16
17 [TMDM](https://www.isotopicmaps.org/sam/sam-model/2006-06-18/) 4.2, 5.1–5.3,
18 5.7–5.8, 6.2, 6.6–6.7 and 7.1–7.2 govern locator identity, canonical same-map
19 merging and Core Subjects. Existing public TMDM operations implement those
20 transformations. The XML ID-uniqueness set contains strings only, never Topics.
21
22 [CXTM](https://www.isotopicmaps.org/cxtm/cxtm.html) 3.2–3.4, 3.10–3.15, 3.18
23 and clause 4 supply the existing numbering, reference, locator and ordering
24 rules. Baselines contain relative locators because CXTM 3.18 removes the given
25 base-locator prefix; stored TMDM identifiers stay absolute. CXTM is not used to
26 infer TMDM identity.
27
28 Local choices: XML events form a temporary source tree, validated before model
29 construction. Deserialization then follows document order, including forward
30 references. Existing replacement handles remain authoritative. Unknown XTM
31 structure is rejected; excluded grammar extension points signal unsupported,
32 without claiming full validation of their unimplemented subgrammars. TMDM
33 conflicts propagate as their original specific conditions, rather than becoming
34 XML errors. No transaction policy or extra merging rule is attributed to XTM.
35
36 ## Public API and diagnostics
37
38 ```lisp
39 (topic-maps.xtm:deserialize-octets octets
40 :document-iri "https://example.invalid/maps/example.xtm"
41 :source-key "caller-supplied:source-label")
42 ```
43
44 Returns a new actual `topic-maps.tmdm:topic-map`. Octets are an unsigned-byte-8
45 vector; no filenames or decoded strings are implicitly accepted. Document IRI
46 is required, absolute and fragment-free. Source-Key affects diagnostics only.
47 Source bytes and source-tree nodes are not mixed into the TMDM information items.
48
49 `xtm-rejection` carries `error-code`, `error-detail`, `error-document-iri`,
50 `error-source-key`, `error-line`, `error-column`. Positions are approximate SAX
51 positions at a start-tag boundary, never invented source spans. `xtm-unsupported`
52 is a separate condition. `xtm-xml-error` retains `error-cause` and distinguishes
53 `:xml-not-well-formed` from `:doctype-forbidden`. Existing TMDM
54 `item-identifier-error` and `core-subject-error` are preserved. API argument/type
55 errors and an unqualified XML runtime are configuration errors, not XTM rejects.
56
57 All actual XML parsing goes through `topic-maps.xml:parse-octets`. Its unchanged
58 19D0B ABI markers, mandatory namespaces, rejecting DOCTYPE/entity policy and
59 strict patched decoders remain required. There are no retrieval callbacks,
60 file opens or network retrieval in the XTM reader. External `topicRef` locators
61 create/reuse Topics without retrieving the referenced document. The pinned
62 CXML XML-1.0 name tables check ID/NCName characters; this helper does not bypass
63 the XML byte boundary. ID whitespace is collapsed and document uniqueness checked.
64
65 ## Original contracts and actual model
66
67 The Document IRI for each input is:
68 `https://xtm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/xtm2/in/<file>`.
69
70 | Unmodified original | Actual TMDM | CXTM bytes |
71 |---|---|---:|
72 | empty.xtm | empty Topic Map | 23 |
73 | topic.xtm | one Topic, II = Document IRI + `#topic` | 114 |
74 | itemid-tm.xtm | empty Map, Map II = Document IRI + `#the-topic-map` | 94 |
75 | topic-type.xtm | five Topics, one Association, two Roles | 963 |
76 | topic-type-duplicate.xtm | same normalized information items | 963 |
77 | merge-three-way.xtm | one Topic with the three absolute `#topic`, `#topic2`, `#topic3` IIs | 168 |
78
79 The separate XTM report records six genuine CTM-independent reader/CXTM
80 comparisons. The remaining 107 positive and 11 negative XTM inputs are unexecuted;
81 they are not imported as passing contracts. These six results do not establish
82 complete XTM conformance.
83
84 The unchanged `ctm/in/mergemap-xtm.sub` is also read directly with its own absolute
85 Document IRI under that same base. Its Topics are the `#lars` and `#person` II
86 Topics plus the three Core SI Topics `.../model/type-instance`, `.../model/type`,
87 `.../model/instance`. One Association has the type-instance Topic as its Type.
88 Its Roles have `(Type, Player)` = `(type, person)` and `(instance, lars)`.
89 All Parent, Type, Player, rolesPlayed and canonical-handle references are checked
90 by actual EQ. This direct fixture is **not** the CTM Mergemap contract.
91
92 Retained Topic, Association and Role handles are checked during later declaration
93 merges. Tests distinguish an SI hit (no new II) from a forward reference which
94 first creates an II and later merges; declaration order therefore can affect
95 identifier sets. Subject Locators are not used by topicRef lookup. Separate
96 Document IRIs preserve different local identities. CXTM leaves both current
97 objects and retained handles unchanged.
98
99 ## Reproduce offline
100
101 No new dependency downloads. Use the already qualified original sources and
102 unchanged local decoder patches:
103
104 ```sh
105 cd /Users/rgb/workspace/topic-maps
106 python3 scripts/validate-slice-19d1.py \
107 --sbcl /nix/store/hlsiblxpxr1hax3dw9x8znd6zy097ddj-sbcl-2.5.4/bin/sbcl \
108 --output /tmp/topic-maps-19d1-review
109 ```
110
111 The driver prepares sources with offline Nix evaluation and verified hashes,
112 then runs XML qualification, two independent XTM processes with byte-identical
113 reports, and the unchanged full regression. SBCL runs under `sandbox-exec` with
114 network denied; there are no loads into persistent images. `execution.json`
115 records commands, times, source hashes, temporary runtime provenance and results.
116 A full offline Nix rebuild is **NOT-RUN**; existing Store sources plus fresh
117 patched-source ASDF loading are tested. XML qualification is not an XTM PASS.
118
119 For inspection, prepare once (the target must not already exist):
120
121 ```sh
122 cd /Users/rgb/workspace/topic-maps
123 xtm_registry=$(python3 scripts/prepare-cxml.py --directory /tmp/xtm-review-sources --registry)
124 /usr/bin/sandbox-exec -p '(version 1)(allow default)(deny network*)' \
125 /usr/bin/env CL_SOURCE_REGISTRY="$xtm_registry" \
126 /nix/store/hlsiblxpxr1hax3dw9x8znd6zy097ddj-sbcl-2.5.4/bin/sbcl \
127 --no-sysinit --no-userinit
128 ```
129
130 In this fresh process:
131
132 ```lisp
133 (require :asdf)
134 (asdf:load-asd #P"/Users/rgb/workspace/topic-maps/topic-maps.asd")
135 (asdf:load-system "topic-maps/xtm")
136 (defparameter *map*
137 (with-open-file (in #P"/Users/rgb/workspace/topic-maps/vendor/cxtm/ctm/in/mergemap-xtm.sub"
138 :element-type '(unsigned-byte 8))
139 (let ((bytes (make-array (file-length in) :element-type '(unsigned-byte 8))))
140 (read-sequence bytes in)
141 (topic-maps.xtm:deserialize-octets bytes
142 :document-iri "https://example.invalid/mergemap-xtm.sub"
143 :source-key "pinned:ctm/in/mergemap-xtm.sub"))))
144 (describe *map*)
145 (defparameter *association* (first (topic-maps.tmdm:topic-map-associations *map*)))
146 (describe *association*)
147 (dolist (role (topic-maps.tmdm:association-roles *association*))
148 (describe role)
149 (describe (topic-maps.tmdm:association-role-player role)))
150 (eq *map* (topic-maps.tmdm:association-parent *association*)) ; T
151 ```
152
153 Not implemented: CTM XTM-Mergemap dispatch, XTM mergeMap, Names, Variants,
154 Occurrences, explicit Associations, Reification and embedded XML values.
155 No HyperDoc changes or additional XTM features belong to this slice.