]> dreyeck.freedombox.rocks Git - topic-maps.git/blob - docs/reading-first-occurrences.md
Add string and URI Occurrences with merging and CXTM validation
[topic-maps.git] / docs / reading-first-occurrences.md
1 # Reading the First Four CTM Occurrences
2
3 Slice 5 starts from the clean, verified Slice 4 commit
4 `8036a2b1ce009218f33d300edcc5f715ec1d1124`, on `codex/ctm-slice-5`.
5 The upstream corpus remains pinned to
6 `cd165b150176c1d6842b37be9cc9051b40d4996b` (CTM draft 2010-03-31).
7
8 ## Expectations established before implementation
9
10 The original sources and CXTM baselines were read before implementation.
11 The common document/base IRI is
12 `https://ctm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/ctm/in/`
13 followed by the input filename. Each local Topic has only Item Identifier
14 `Document-IRI#identifier`; Subject Identifiers/Locators are empty.
15 Occurrence item identifiers start empty. Each map contains no Names or Associations.
16 The XML datatypes are locators, not additional Topic items.
17
18 | Original case | Expected live items | Expected Occurrence properties | Baseline bytes |
19 | --- | --- | --- | ---: |
20 | `occurrence-string.ctm` | Topic Map; Topics `topic`, `occtype`; one Occurrence | Parent `topic`, Type `occtype`, Value `Occurrence`, Datatype `http://www.w3.org/2001/XMLSchema#string` | 359 |
21 | `occurrence-string2.ctm` | Same items/properties; no trailing semicolon | Same properties | 359 |
22 | `occurrence-uri-relative.ctm` | Topic Map; Topics `topic`, `occtype`; one Occurrence | Parent `topic`, Type `occtype`, Value absolute corpus IRI ending `/ctm/in/file.html`, Datatype `http://www.w3.org/2001/XMLSchema#anyURI` | 358 |
23 | `occurrence-two.ctm` | Topic Map; Topics `tmcl-schema`, `description`, `see-also`; two Occurrences | Parent `tmcl-schema`; Types `description`/`see-also`; multiline string and absolute HTTP locator | 730 |
24
25 The standard supplies these expectations; baseline XML independently supplies
26 relative locator spellings, canonical numbering and output bytes. The baseline
27 alone cannot tell whether a relative URI value was stored or an absolute value
28 was correctly normalized. Direct semantic tests must establish that distinction.
29
30 ## Normative clauses
31
32 - [CTM](https://www.isotopicmaps.org/ctm/ctm.html): §§3.2, 3.3.3–3.3.7,
33 3.3.11, 3.4.1–3.4.2, 3.8–3.10.
34 - [TMDM](https://www.isotopicmaps.org/sam/sam-model/2006-06-18/): §§4.3,
35 5.1, 5.3, 5.6, 6.2, 6.5.
36 - [CXTM](https://www.isotopicmaps.org/cxtm/cxtm.html): §§3.1, 3.5, 3.8,
37 3.11–3.23 and 4.2–4.6, 4.9 (string/anyURI subset only).
38
39 CTM Type denotes a Topic. A literal assigns value/datatype; an IRI-form literal
40 uses IRI creation, including relative resolution. TMDM's IRI datatype requires
41 absolute locators. A quoted string is not passed through IRI resolution merely
42 because it resembles a URI. Explicit datatype qualifiers are recognized but
43 remain unsupported in this bounded reader.
44
45 An Occurrence is distinct from its Type Topic and its value. Its equality uses
46 value, datatype, type, scope and parent. Supported scope is empty and reifier
47 null. Merging equal Occurrences produces a replacement and unions item identifiers.
48 CXTM orders Occurrences by value, datatype, type, scope and parent; the value
49 is locator-normalized only for anyURI, while datatype output is not base-stripped.
50
51 ## Original, unmodified sources
52
53 `occurrence-string.ctm` (no final newline):
54
55 ```ctm
56 topic
57 occtype: "Occurrence";.
58 ```
59
60 `occurrence-string2.ctm` (no final newline):
61
62 ```ctm
63 topic
64 occtype: "Occurrence".
65 ```
66
67 `occurrence-uri-relative.ctm` (initial/final newline):
68
69 ```ctm
70
71 topic occtype: <file.html> .
72 ```
73
74 `occurrence-two.ctm` (initial/final newline; indentation inside the string matters):
75
76 ```ctm
77
78 tmcl-schema
79 description: "A meta-schema for TMCL, describing the allowed structure
80 of TMCL schemas.";
81 see-also: http://www.isotopicmaps.org/tmcl/tmcl.html .
82 ```
83
84 ## Actual objects and API contract
85
86 `occurrence` is an ordinary TMDM DEFSTRUCT. Its materialized slots are value,
87 datatype, type, parent, item-identifiers and the private forwarding link.
88 Datatype locators use the model's existing string locator representation.
89 Scope is implicitly empty; reifier is implicitly null. No source object or XML-only
90 Occurrence model is introduced.
91
92 ```lisp
93 (topic-maps.tmdm:add-occurrence parent-topic type-topic value
94 :datatype (topic-maps.tmdm:string-datatype) :item-identifiers nil)
95 (topic-maps.tmdm:topic-occurrences topic)
96 (topic-maps.tmdm:canonical-occurrence retained-handle)
97 (topic-maps.tmdm:occurrence-parent occurrence)
98 (topic-maps.tmdm:occurrence-type occurrence)
99 (topic-maps.tmdm:occurrence-value occurrence)
100 (topic-maps.tmdm:occurrence-datatype occurrence)
101 (topic-maps.tmdm:occurrence-item-identifiers occurrence)
102 ```
103
104 `add-occurrence` validates before mutation, accepts string or already absolute
105 anyURI values, and performs duplicate replacement. It never resolves strings
106 against a Document IRI. Type and Parent must belong to the same map. Optional
107 nonempty item identifiers are supported through the model API and canonicalizer;
108 CTM does not assign them to Occurrences. Unequal items cannot share an Item
109 Identifier. Conflicts requiring Topic/Occurrence reification are explicitly
110 rejected; reification is not implemented as a side effect.
111
112 A replaced handle is not EQ to its replacement. Its canonical representative is:
113
114 ```lisp
115 (eq old-occurrence new-occurrence) ; NIL after replacement
116 (eq (topic-maps.tmdm:canonical-occurrence old-occurrence)
117 new-occurrence) ; T
118 ```
119
120 Public property readers resolve the Occurrence handle and return current stored
121 parent/type references. Topic merges rewrite those references in the live model;
122 readers do not hide stale Topic references by lazily canonicalizing them. Collection
123 readers return copies. CXTM neither merges nor compresses old handle chains.
124
125 The reported insertion ordinals are **not** CXTM numbers. In each of the first
126 three maps, actual Topic T1 is `topic`, T2 is `occtype`, and O1 has Parent EQ T1,
127 Type EQ T2. Its Item Identifiers are NIL. CXTM sorts T2 before T1, so the occurrence
128 appears in CXTM Topic 2 and references Topic 1 as its type.
129
130 In `occurrence-two.ctm`, actual T1 is `tmcl-schema`, T2 is `description`, T3 is
131 `see-also`. O1/O2 have Parent EQ T1; Types EQ T2/T3 respectively. O1's exact value
132 is `A meta-schema for TMCL, describing the allowed structure`, then LF, then
133 **20 spaces**, then `of TMCL schemas.`. O2's exact value is
134 `http://www.isotopicmaps.org/tmcl/tmcl.html`. Datatypes are xsd:string and xsd:anyURI;
135 both identifier sets are NIL. CXTM sorts Topics description=1, see-also=2,
136 tmcl-schema=3, and numbers the two Occurrences 1 and 2 within Topic 3.
137
138 For the relative URI case the actual O1 value is:
139
140 ```
141 https://ctm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/ctm/in/file.html
142 ```
143
144 That is a formed locator, not the raw source spelling `<file.html>`. The
145 original source spelling is retained separately in the evidence report. The
146 source has not been edited to force a baseline match. Different Document IRIs
147 still give local Topics different Item Identifiers; matching CXTM under different
148 bases is not a cross-document Subject Identity assertion.
149
150 ## Executable inspection
151
152 In a fresh Common Lisp process:
153
154 ```lisp
155 (require :asdf)
156 (asdf:initialize-source-registry '(:source-registry :ignore-inherited-configuration))
157 (asdf:load-asd #p"/Users/rgb/workspace/topic-maps/topic-maps.asd")
158 (asdf:load-system "topic-maps/ctm")
159 (let* ((iri "https://ctm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/ctm/in/occurrence-string.ctm")
160 (map (topic-maps.ctm:deserialize-file
161 (asdf:system-relative-pathname "topic-maps/ctm" "vendor/cxtm/ctm/in/occurrence-string.ctm")
162 :document-iri iri))
163 (parent (first (topic-maps.tmdm:topic-map-topics map)))
164 (item (first (topic-maps.tmdm:topic-occurrences parent))))
165 (assert (eq parent (topic-maps.tmdm:occurrence-parent item)))
166 (assert (eq (second (topic-maps.tmdm:topic-map-topics map))
167 (topic-maps.tmdm:occurrence-type item)))
168 (describe item)
169 (describe (topic-maps.tmdm:occurrence-type item)))
170 ```
171
172 ## Implementation and limitations
173
174 - `src/tmdm/items.lisp`: Occurrence structure, canonical handle/readers.
175 - `src/tmdm/occurrences.lisp:41`: duplicate replacement and identifier union;
176 `:55`: public construction and validation.
177 - `src/tmdm/merging.lisp`: property union and stored parent/type rewrites.
178 - `src/ctm/reader.lisp:47`: IRI locators without Topic creation;
179 `:87`: literal semantics and errors; `:123`: Type Topic/reference mapping.
180 - `src/cxtm/canonicalizer.lisp:51`: value-kind and Occurrence comparison;
181 serializer emits value, datatype, type and identifiers after Names, before roles-played.
182 - `tests/ctm-occurrences.lisp`: original sources, direct object/merge tests and
183 independent construction-order variants. Python independently checks the
184 actual properties and raw baseline bytes.
185
186 The existing URI lexer/QName rules are retained. Use layout where a colon would
187 otherwise form a QName (for example before an undelimited URI literal).
188 Prefix directives, triple-quoted strings, explicit `^^` datatype-qualified CTM
189 literals, numeric/date values, nonempty Scope, Reification, Variants, Templates,
190 cross-document merging and HyperDoc integration remain unsupported. A missing
191 Occurrence value is CTM-REJECTION; recognized excluded literals are CTM-UNSUPPORTED;
192 unbound QNames preserve the specific UNDEFINED-PREFIX rejection.
193
194 There is no full IRI grammar validator in the model API: locator arguments must
195 already be validated/resolved. CXTM's Unicode 15.1.0 bound and explicit rejection
196 of ambiguous normalized Topic keys remain unchanged. The documented Slice 4
197 root-LF interpretation is retained. All four selected cases avoid excluded features.