]> dreyeck.freedombox.rocks Git - topic-maps.git/blob - docs/reading-first-occurrences.md
perf(tmdm): index association normalization merge candidates
[topic-maps.git] / docs / reading-first-occurrences.md
1 # Reading the First Four CTM Occurrences
2
3 Historical Slice 5 subset. Slice 6 supersedes the exclusions of prefixes and
4 explicit quoted-string datatype qualifiers below; see
5 [Reading Prefixes and Explicit Datatypes](reading-prefixes-and-datatypes.md).
6
7
8 Slice 5 starts from the clean, verified Slice 4 commit
9 `8036a2b1ce009218f33d300edcc5f715ec1d1124`, on `codex/ctm-slice-5`.
10 The upstream corpus remains pinned to
11 `cd165b150176c1d6842b37be9cc9051b40d4996b` (CTM draft 2010-03-31).
12
13 ## Expectations established before implementation
14
15 The original sources and CXTM baselines were read before implementation.
16 The common document/base IRI is
17 `https://ctm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/ctm/in/`
18 followed by the input filename. Each local Topic has only Item Identifier
19 `Document-IRI#identifier`; Subject Identifiers/Locators are empty.
20 Occurrence item identifiers start empty. Each map contains no Names or Associations.
21 The XML datatypes are locators, not additional Topic items.
22
23 | Original case | Expected live items | Expected Occurrence properties | Baseline bytes |
24 | --- | --- | --- | ---: |
25 | `occurrence-string.ctm` | Topic Map; Topics `topic`, `occtype`; one Occurrence | Parent `topic`, Type `occtype`, Value `Occurrence`, Datatype `http://www.w3.org/2001/XMLSchema#string` | 359 |
26 | `occurrence-string2.ctm` | Same items/properties; no trailing semicolon | Same properties | 359 |
27 | `occurrence-uri-relative.ctm` | Topic Map; Topics `topic`, `occtype`; one Occurrence | Parent `topic`, Type `occtype`, Value absolute corpus IRI ending `/ctm/in/file.html`, Datatype `http://www.w3.org/2001/XMLSchema#anyURI` | 358 |
28 | `occurrence-two.ctm` | Topic Map; Topics `tmcl-schema`, `description`, `see-also`; two Occurrences | Parent `tmcl-schema`; Types `description`/`see-also`; multiline string and absolute HTTP locator | 730 |
29
30 The standard supplies these expectations; baseline XML independently supplies
31 relative locator spellings, canonical numbering and output bytes. The baseline
32 alone cannot tell whether a relative URI value was stored or an absolute value
33 was correctly normalized. Direct semantic tests must establish that distinction.
34
35 ## Normative clauses
36
37 - [CTM](https://www.isotopicmaps.org/ctm/ctm.html): §§3.2, 3.3.3–3.3.7,
38 3.3.11, 3.4.1–3.4.2, 3.8–3.10.
39 - [TMDM](https://www.isotopicmaps.org/sam/sam-model/2006-06-18/): §§4.3,
40 5.1, 5.3, 5.6, 6.2, 6.5.
41 - [CXTM](https://www.isotopicmaps.org/cxtm/cxtm.html): §§3.1, 3.5, 3.8,
42 3.11–3.23 and 4.2–4.6, 4.9 (string/anyURI subset only).
43
44 CTM Type denotes a Topic. A literal assigns value/datatype; an IRI-form literal
45 uses IRI creation, including relative resolution. TMDM's IRI datatype requires
46 absolute locators. A quoted string is not passed through IRI resolution merely
47 because it resembles a URI. Explicit datatype qualifiers are recognized but
48 remain unsupported in this bounded reader.
49
50 An Occurrence is distinct from its Type Topic and its value. Its equality uses
51 value, datatype, type, scope and parent. Supported scope is empty and reifier
52 null. Merging equal Occurrences produces a replacement and unions item identifiers.
53 CXTM orders Occurrences by value, datatype, type, scope and parent; the value
54 is locator-normalized only for anyURI, while datatype output is not base-stripped.
55
56 ## Original, unmodified sources
57
58 `occurrence-string.ctm` (no final newline):
59
60 ```ctm
61 topic
62 occtype: "Occurrence";.
63 ```
64
65 `occurrence-string2.ctm` (no final newline):
66
67 ```ctm
68 topic
69 occtype: "Occurrence".
70 ```
71
72 `occurrence-uri-relative.ctm` (initial/final newline):
73
74 ```ctm
75
76 topic occtype: <file.html> .
77 ```
78
79 `occurrence-two.ctm` (initial/final newline; indentation inside the string matters):
80
81 ```ctm
82
83 tmcl-schema
84 description: "A meta-schema for TMCL, describing the allowed structure
85 of TMCL schemas.";
86 see-also: http://www.isotopicmaps.org/tmcl/tmcl.html .
87 ```
88
89 ## Actual objects and API contract
90
91 `occurrence` is an ordinary TMDM DEFSTRUCT. Its materialized slots are value,
92 datatype, type, parent, item-identifiers and the private forwarding link.
93 Datatype locators use the model's existing string locator representation.
94 Scope is implicitly empty; reifier is implicitly null. No source object or XML-only
95 Occurrence model is introduced.
96
97 ```lisp
98 (topic-maps.tmdm:add-occurrence parent-topic type-topic value
99 :datatype (topic-maps.tmdm:string-datatype) :item-identifiers nil)
100 (topic-maps.tmdm:topic-occurrences topic)
101 (topic-maps.tmdm:canonical-occurrence retained-handle)
102 (topic-maps.tmdm:occurrence-parent occurrence)
103 (topic-maps.tmdm:occurrence-type occurrence)
104 (topic-maps.tmdm:occurrence-value occurrence)
105 (topic-maps.tmdm:occurrence-datatype occurrence)
106 (topic-maps.tmdm:occurrence-item-identifiers occurrence)
107 ```
108
109 `add-occurrence` validates before mutation, accepts string or already absolute
110 anyURI values, and performs duplicate replacement. It never resolves strings
111 against a Document IRI. Type and Parent must belong to the same map. Optional
112 nonempty item identifiers are supported through the model API and canonicalizer;
113 CTM does not assign them to Occurrences. Unequal items cannot share an Item
114 Identifier. Conflicts requiring Topic/Occurrence reification are explicitly
115 rejected; reification is not implemented as a side effect.
116
117 A replaced handle is not EQ to its replacement. Its canonical representative is:
118
119 ```lisp
120 (eq old-occurrence new-occurrence) ; NIL after replacement
121 (eq (topic-maps.tmdm:canonical-occurrence old-occurrence)
122 new-occurrence) ; T
123 ```
124
125 Public property readers resolve the Occurrence handle and return current stored
126 parent/type references. Topic merges rewrite those references in the live model;
127 readers do not hide stale Topic references by lazily canonicalizing them. Collection
128 readers return copies. CXTM neither merges nor compresses old handle chains.
129
130 The reported insertion ordinals are **not** CXTM numbers. In each of the first
131 three maps, actual Topic T1 is `topic`, T2 is `occtype`, and O1 has Parent EQ T1,
132 Type EQ T2. Its Item Identifiers are NIL. CXTM sorts T2 before T1, so the occurrence
133 appears in CXTM Topic 2 and references Topic 1 as its type.
134
135 In `occurrence-two.ctm`, actual T1 is `tmcl-schema`, T2 is `description`, T3 is
136 `see-also`. O1/O2 have Parent EQ T1; Types EQ T2/T3 respectively. O1's exact value
137 is `A meta-schema for TMCL, describing the allowed structure`, then LF, then
138 **20 spaces**, then `of TMCL schemas.`. O2's exact value is
139 `http://www.isotopicmaps.org/tmcl/tmcl.html`. Datatypes are xsd:string and xsd:anyURI;
140 both identifier sets are NIL. CXTM sorts Topics description=1, see-also=2,
141 tmcl-schema=3, and numbers the two Occurrences 1 and 2 within Topic 3.
142
143 For the relative URI case the actual O1 value is:
144
145 ```
146 https://ctm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/ctm/in/file.html
147 ```
148
149 That is a formed locator, not the raw source spelling `<file.html>`. The
150 original source spelling is retained separately in the evidence report. The
151 source has not been edited to force a baseline match. Different Document IRIs
152 still give local Topics different Item Identifiers; matching CXTM under different
153 bases is not a cross-document Subject Identity assertion.
154
155 ## Executable inspection
156
157 In a fresh Common Lisp process:
158
159 ```lisp
160 (require :asdf)
161 (asdf:initialize-source-registry '(:source-registry :ignore-inherited-configuration))
162 (asdf:load-asd #p"/Users/rgb/workspace/topic-maps/topic-maps.asd")
163 (asdf:load-system "topic-maps/ctm")
164 (let* ((iri "https://ctm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/ctm/in/occurrence-string.ctm")
165 (map (topic-maps.ctm:deserialize-file
166 (asdf:system-relative-pathname "topic-maps/ctm" "vendor/cxtm/ctm/in/occurrence-string.ctm")
167 :document-iri iri))
168 (parent (first (topic-maps.tmdm:topic-map-topics map)))
169 (item (first (topic-maps.tmdm:topic-occurrences parent))))
170 (assert (eq parent (topic-maps.tmdm:occurrence-parent item)))
171 (assert (eq (second (topic-maps.tmdm:topic-map-topics map))
172 (topic-maps.tmdm:occurrence-type item)))
173 (describe item)
174 (describe (topic-maps.tmdm:occurrence-type item)))
175 ```
176
177 ## Implementation and limitations
178
179 - `src/tmdm/items.lisp`: Occurrence structure, canonical handle/readers.
180 - `src/tmdm/occurrences.lisp:41`: duplicate replacement and identifier union;
181 `:55`: public construction and validation.
182 - `src/tmdm/merging.lisp`: property union and stored parent/type rewrites.
183 - `src/ctm/reader.lisp:47`: IRI locators without Topic creation;
184 `:87`: literal semantics and errors; `:123`: Type Topic/reference mapping.
185 - `src/cxtm/canonicalizer.lisp:51`: value-kind and Occurrence comparison;
186 serializer emits value, datatype, type and identifiers after Names, before roles-played.
187 - `tests/ctm-occurrences.lisp`: original sources, direct object/merge tests and
188 independent construction-order variants. Python independently checks the
189 actual properties and raw baseline bytes.
190
191 The existing URI lexer/QName rules are retained. Use layout where a colon would
192 otherwise form a QName (for example before an undelimited URI literal).
193 Prefix directives, triple-quoted strings, explicit `^^` datatype-qualified CTM
194 literals, numeric/date values, nonempty Scope, Reification, Variants, Templates,
195 cross-document merging and HyperDoc integration remain unsupported. A missing
196 Occurrence value is CTM-REJECTION; recognized excluded literals are CTM-UNSUPPORTED;
197 unbound QNames preserve the specific UNDEFINED-PREFIX rejection.
198
199 There is no full IRI grammar validator in the model API: locator arguments must
200 already be validated/resolved. CXTM's Unicode 15.1.0 bound and explicit rejection
201 of ambiguous normalized Topic keys remain unchanged. The documented Slice 4
202 root-LF interpretation is retained. All four selected cases avoid excluded features.