]> dreyeck.freedombox.rocks Git - topic-maps.git/blob - docs/reading-prefixes-and-datatypes.md
Canonicalize DateTime timezone offsets with explicit baseline discrepancies
[topic-maps.git] / docs / reading-prefixes-and-datatypes.md
1 # Reading Prefixes and Explicit Datatypes
2
3 Historical Slice 6 subset. Slice 7 adds integer literals and xsd:integer CXTM
4 canonicalization, superseding the integer exclusions below. See
5 [Reading Integer Occurrences](reading-integer-occurrences.md).
6
7
8 Slice 6 starts at `437ad53ea7539d36db3fa871d0931650b7a6a03e`. A prefix is a
9 source abbreviation for an IRI, not a Topic. An Occurrence datatype is a locator;
10 its Type is a Topic reference. These two properties remain different even if an
11 author deliberately uses the same IRI for both.
12
13 ## Normative requirements
14
15 [ISO/IEC 13250-6:2010](https://www.isotopicmaps.org/ctm/ctm.html):
16 §3.2 processes components in document order. §§3.3.3–3.3.5 form IRIs and resolve
17 QNames by concatenation; undefined prefixes are errors. §3.16.1 binds an
18 identifier to an IRI and permits repeat binding only to the same reference.
19 Its reference production accepts an IRI, not another QName. §§3.4.1–3.4.2 and
20 3.10 assign the quoted/escape-decoded lexical value and the explicit datatype
21 to the Occurrence. A qualifier is `^^` followed by an IRI reference.
22
23 [TMDM 2006](https://www.isotopicmaps.org/sam/sam-model/2006-06-18/):
24 §4.3 defines string/locator equality. §4.4 permits other datatypes and stores
25 value representations separately from datatype locators. A datatype Topic need
26 not exist. `xsd:anyURI` content must be absolute. §5.6 defines Occurrence
27 properties/equality; §6.5 merges equal Occurrences and unions their identifiers.
28 The existing empty-Scope/null-Reifier subset and replacement handles continue.
29
30 [ISO/IEC 13250-4:2009](https://www.isotopicmaps.org/cxtm/cxtm.html):
31 §3.14 writes the datatype locator without base stripping. §3.16 uses locator
32 normalization for anyURI, special lexical rules for integer/decimal/date/dateTime,
33 and the string rule for Other. §3.20 applies NFC to encoded string properties.
34 §§4.2–4.5 and 4.9 order Occurrences by value, datatype, Type, Scope, Parent.
35 The serializer's existing §§3.8, 3.18–3.19 numbering/base rules remain unchanged.
36
37 ## Original inputs and expected items
38
39 The upstream corpus is unchanged at
40 `cd165b150176c1d6842b37be9cc9051b40d4996b`. It targets draft 2010-03-31;
41 the published standard is authoritative. No conflict blocks these seven cases.
42 The following expectations were established from those clauses and checked
43 against the original baselines before implementation.
44
45 Every case uses the absolute Document IRI formed by concatenating
46 `https://ctm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/`
47 and its full `ctm/in/...` or `ctm/invalid/...` case ID. This IRI is also the CXTM
48 Base Locator. It is independent of the physical checkout. Model Item Identifiers
49 remain absolute; the baselines' `#topic`/`#occtype` follow CXTM §3.18. Datatype
50 locators remain full IRIs under §3.14.
51
52 ### ctm/in/occurrence-explicit-datatype.ctm
53
54 ```ctm
55 %prefix xsd http://www.w3.org/2001/XMLSchema#
56
57 topic
58 occtype: "Occurrence"^^xsd:string;.
59 ```
60
61 The original source has no final LF; the display fence adds only a visual line break.
62
63 Two Topics, `topic` and `occtype`, each with its document-local Item Identifier. One Occurrence O1: Parent EQ topic, Type EQ occtype, Value `Occurrence`, Datatype `http://www.w3.org/2001/XMLSchema#string`. No Topic for xsd or its datatype. Empty SI/SL and Occurrence identifiers. Original CXTM: 359 bytes.
64
65 ### ctm/in/occurrence-datatype-unknown.ctm
66
67 ```ctm
68 topic
69 occtype: "BOGUS"^^http://example.com/bogus.
70 ```
71
72 The same two Topic identities. One Occurrence O1: Parent EQ topic, Type EQ occtype, Value `BOGUS`, Datatype `http://example.com/bogus`. The unknown datatype is a locator, not a third Topic. Original CXTM: 339 bytes.
73
74 ### ctm/in/prefix-duplicate.ctm
75
76 ```ctm
77 %prefix ex http://psi.example.org/
78
79 %prefix ex http://psi.example.org/
80
81 ex:fake.
82 ```
83
84 One Topic with SI `http://psi.example.org/fake`, no II or SL. The identical rebinding is permitted and creates no information item. Original CXTM: 141 bytes.
85
86 ### ctm/in/prefix-duplicate2.ctm
87
88 ```ctm
89 %prefix ex http://psi.example.org/
90
91 ex:fake.
92
93 %prefix ex http://psi.example.org/
94
95 ex:fake2.
96 ```
97
98 Two Topics with SIs `http://psi.example.org/fake` and `http://psi.example.org/fake2`; no II/SL. A binding persists across intervening Topic declarations. Original CXTM: 260 bytes.
99
100 ### ctm/in/prefix-multiline.ctm
101
102 ```ctm
103 %prefix
104 ex
105 http://psi.example.org/ ex:fake.
106 ```
107
108 One Topic with SI `http://psi.example.org/fake`, no II/SL. The directive is token based and may span lines; the next Topic can follow its IRI on the same line. Original CXTM: 141 bytes.
109
110 ### ctm/invalid/prefix-collision.ctm
111
112 ```ctm
113 %prefix ex http://www.example.org/
114 %prefix ex http://www.example.net/
115
116 ```
117
118 Reject the second binding: `PREFIX-COLLISION`, a `CTM-REJECTION`, code `:PREFIX-COLLISION`, zero-based offset 43, line 2, column 9 (the second prefix identifier). No map is returned.
119
120 ### ctm/invalid/prefix-undefined2.ctm
121
122 ```ctm
123 #
124 # In one of the CTM drafts, xsd was bound to <http://www.w3.org/2001/XMLSchema#>
125 # This binding was removed from CTM
126 #
127
128 foo occ: "1"^^xsd:int .
129 ```
130
131 Reject `xsd:int`: `UNDEFINED-PREFIX`, a `CTM-REJECTION`, code `:UNDEFINED-PREFIX`, zero-based offset 136, line 6, column 15. There is no implicit xsd binding. The failure precedes any CXTM datatype processing. No map is returned.
132
133 ## Observed Lisp objects and results
134
135 The two Occurrence examples each return a `TOPIC-MAP` with two `TOPIC` structures
136 in source order: T1 owner, T2 type. T1's `topic-occurrences` contains O1, an
137 `OCCURRENCE` structure. O1's raw stored parent/type references are EQ T1/T2.
138 Its value/datatype are exactly the strings stated above; its Item Identifiers
139 are NIL. The owner/type Topics have II `DocumentIRI#topic`/`DocumentIRI#occtype`,
140 SI/SL NIL. No Names or Associations occur. Canonical topic numbers are type=1,
141 owner=2, with Occurrence number 1 inside owner.
142
143 The three positive prefix-only examples return one/two Topics in source order,
144 with precisely the SI sets above and empty II/SL/Names/Occurrences. Their
145 canonical numbers are fake=1, fake2=2 where applicable. The two negative cases
146 produce the specific conditions above, never I/O/generic failures.
147
148 These local observations are separately recorded in `reports/slice-6/prefixes.json`;
149 the Python checker independently checks source bytes, ordinal references, values,
150 datatypes and negative source positions. The actual CXTM UTF-8 bytes are compared
151 without any preprocessing to the original upstream baselines. All five comparisons
152 and both rejections pass. The preceding 28 canonical outputs remain byte-identical.
153 See `slice-6-validation.md` for full regression counts and fresh-process commands.
154
155 ## Public API and inspection
156
157 Load `topic-maps/ctm` for deserialization and `topic-maps/cxtm` for canonicalization.
158 No API signature changes:
159
160 - `deserialize-string text :document-iri absolute-iri :source-name optional-name`
161 - `deserialize-file pathname :document-iri absolute-iri :source-name optional-name`
162 - `add-occurrence parent-topic type-topic value :datatype absolute-locator :item-identifiers ids`
163 - `occurrence-value`, `occurrence-datatype`, `occurrence-parent`, `occurrence-type`,
164 `occurrence-item-identifiers`, `canonical-occurrence`
165 - `canonicalize`, `canonicalize-octets`, `write-cxtm`, with explicit `:base-locator`.
166
167 The new exported `prefix-collision` condition subclasses `ctm-rejection`, as does
168 `undefined-prefix`. Both are distinct from `ctm-unsupported`. Errors expose code,
169 source name, zero-based character offset and one-based line/column.
170
171 ```lisp
172 (require :asdf)
173 (asdf:initialize-source-registry '(:source-registry :ignore-inherited-configuration))
174 (asdf:load-asd #p"/Users/rgb/workspace/topic-maps/topic-maps.asd")
175 (asdf:load-system "topic-maps/ctm")
176 (asdf:load-system "topic-maps/cxtm")
177 (let* ((doc "https://example.invalid/inspection.ctm")
178 (map (topic-maps.ctm:deserialize-string
179 "%prefix ex <urn:example:> owner ex:description: \"00042\"^^ex:custom."
180 :document-iri doc))
181 (owner (first (topic-maps.tmdm:topic-map-topics map)))
182 (item (first (topic-maps.tmdm:topic-occurrences owner))))
183 (assert (eq owner (topic-maps.tmdm:occurrence-parent item)))
184 (assert (string= "00042" (topic-maps.tmdm:occurrence-value item)))
185 (assert (string= "urn:example:custom" (topic-maps.tmdm:occurrence-datatype item)))
186 (assert (equal '("urn:example:description")
187 (topic-maps.tmdm:topic-subject-identifiers
188 (topic-maps.tmdm:occurrence-type item))))
189 (describe item)
190 (describe (topic-maps.tmdm:occurrence-type item))
191 (write-string (topic-maps.cxtm:canonicalize map :base-locator doc)))
192 ```
193
194 ## Implementation boundaries
195
196 Bindings are parser-local, case-sensitive and document-ordered. A successful or
197 failed read cannot export a prefix to another read. Prefix IRIs are formed once;
198 QName concatenation neither decodes nor resolves the bound IRI again.
199
200 Quoted strings preserve their escape-decoded lexical values in TMDM. Datatype
201 IRIs/QNames are resolved as locators and never automatically interned as Topics.
202 Unknown datatypes use CXTM's Other string rule, including normal XML escaping/NFC
203 at output. Their value spaces are not interpreted or validated. No implicit
204 numeric, whitespace, URI-content or date conversion is performed.
205
206 Explicit integer/decimal/date/dateTime datatype locators can be stored through
207 the general quoted-string qualifier/API, but CXTM reports
208 `CXTM-UNSUPPORTED :UNSUPPORTED-OCCURRENCE-DATATYPE` for these four exact IRIs;
209 §3.16 requires canonical lexical rules outside this slice. Numeric/date sugar
210 still reports `CTM-UNSUPPORTED`. A quoted relative anyURI value is specifically
211 rejected (`:RELATIVE-URI-VALUE`) under TMDM §4.4, never silently resolved; IRI-form
212 values retain the previously implemented CTM IRI-reference behavior.
213
214 Nonempty Scope, Reification, Variants, Templates, includes/mergemap, cross-document
215 merging, non-UTF-8 encoding declarations and HyperDoc integration remain unsupported.
216 Typed Names and triple-quoted strings remain unsupported. The inherited locator
217 API expects validated/resolved arguments; its scheme check is not a complete RFC
218 3987 validator. The established Unicode 15.1 and normalized Topic-key ambiguity
219 limits, and the previously documented CXTM root-LF interpretation, remain.