]> dreyeck.freedombox.rocks Git - topic-maps.git/blob - docs/reading-decimal-occurrences.md
perf(tmdm): index association normalization merge candidates
[topic-maps.git] / docs / reading-decimal-occurrences.md
1 # Reading Decimal Occurrences
2
3 ## Rules established before implementation
4
5 Base: accepted Slice 7 `17dd58775e25ab495e3c83deb4b5a36f5e5a124b`.
6 The original seven CTM files and CXTM baselines were read before implementation.
7 The unchanged corpus pin is `cd165b150176c1d6842b37be9cc9051b40d4996b`;
8 it targets draft 2010-03-31, while the published standards remain authoritative.
9
10 - [CTM §3.4.1, productions 28–34](https://www.isotopicmaps.org/ctm/ctm.html):
11 a decimal has a fractional digit sequence, optionally an integer component
12 and sign. Decimal and Integer have separate implicit datatype locators.
13 Thus `1.0.`, `.3.` and `9.3.` contain a decimal then a topic terminator;
14 `1.` contains an Integer then the terminator. A qualified quoted string uses
15 its specified datatype. The existing reader policy maps bare numeric values
16 to canonical strings; this is a representation choice, not a claim that CTM
17 mandates early canonicalization. Quoted strings retain their decoded spelling.
18 - [TMDM §§4.3–4.4, 5.6, 6.5](https://www.isotopicmaps.org/sam/sam-model/2006-06-18/):
19 Occurrence [value] is a Unicode string; [datatype] is a separate locator,
20 not a Topic. Equality compares the stored string, datatype, type, scope and
21 parent. The model API preserves valid strings. Mathematical equivalence alone
22 does not merge different stored strings. Duplicate replacement and retained
23 handles keep their existing contract.
24 - [CXTM §§3.14, 3.16, 4.2–4.3, 4.9](https://www.isotopicmaps.org/cxtm/cxtm.html):
25 output uses the decimal's canonical lexical representation without changing
26 the model. Ordering compares stored [value] strings, followed by datatype,
27 Type, Scope and Parent; it does not sort numeric values or reformatted output.
28 - [XML Schema Part 2 §§3.2.3–3.2.3.2, 4.3.6](https://www.w3.org/TR/2004/REC-xmlschema-2-20041028/#decimal):
29 decimal values have finite decimal expansions; precision is not a separate
30 value-space distinction. Lexical forms use ASCII digits and an optional sign;
31 XSD also permits `1` and `1.` in explicitly typed strings. Canonical form has
32 no plus, no unnecessary leading/trailing zeros, and at least one digit on
33 each side of the required decimal point. Zero becomes `0.0`, including
34 negative zero. XML whitespace at the ends is collapsed for datatype
35 interpretation; the stored qualified string retains it. Internal whitespace,
36 exponent notation and Unicode lookalike digits are invalid for this datatype.
37
38 Implementation must use pure character processing: no binary floating point,
39 no Lisp reader/EVAL, no precision-dependent rounding, and no numeric-based
40 change to TMDM equality. The existing single numeric token path is extended.
41 Date, DateTime, CTM `*` semantics, Scope, Reification, Variants, Templates,
42 cross-document merging and HyperDoc integration remain outside this slice.
43
44 ## Actual information items and original baseline results
45
46 Every selected document produces one Topic Map, two Topics (`topic`, `occtype`)
47 and one Occurrence. Each Topic has just its absolute local Item Identifier
48 `DocumentIRI#identifier`; SI/SL sets, Names and Associations are empty. The
49 Occurrence's stored Parent is EQ the owner Topic; Type is EQ `occtype`. Its
50 Item Identifiers are empty. Datatype is always the locator
51 `http://www.w3.org/2001/XMLSchema#decimal`; no datatype Topic is synthesized.
52
53 Document IRI and CXTM Base Locator use the existing deterministic policy:
54 `https://ctm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/`
55 plus the full `ctm/in/...ctm` ID. Stored identifiers remain absolute; baseline
56 `#topic` / `#occtype` output follows the retained CXTM §3.18 policy.
57 CXTM numbers Type Topic=1, owner Topic=2, Occurrence=1, type topicref=1.
58
59 | Original file | Actual stored [value] | Canonical value | Original CXTM bytes |
60 | --- | --- | --- | ---: |
61 | occurrence-decimal.ctm | `"9.3"` | `9.3` | 353 |
62 | occurrence-decimal-dot.ctm | `"0.3"` | `0.3` | 353 |
63 | occurrence-decimal-dot-neg.ctm | `"-0.3"` | `-0.3` | 354 |
64 | occurrence-decimal-leading-zeros.ctm | `"0.3"` | `0.3` | 353 |
65 | occurrence-decimal-minus-sign.ctm | `"-9.3"` | `-9.3` | 354 |
66 | occurrence-decimal-plus-sign.ctm | `"9.3"` | `9.3` | 353 |
67 | occurrence-decimal-1.0.ctm | `"1.0"` | `1.0` | 353 |
68
69 All seven original baselines match byte-for-byte; no XML reparse, trimming,
70 value rewriting or baseline adjustment is used for the comparison.
71 The following are the actual unchanged sources; display fences add a visual
72 line ending where the original has none. Exact sources and object observations
73 are also recorded in `reports/slice-8/decimals.json`.
74
75 ### ctm/in/occurrence-decimal.ctm
76
77 ```ctm
78 topic
79 occtype: 9.3.
80 ```
81
82 Original final LF absent.
83
84 ### ctm/in/occurrence-decimal-dot.ctm
85
86 ```ctm
87 topic
88 occtype: .3.
89 ```
90
91 Original final LF absent.
92
93 ### ctm/in/occurrence-decimal-dot-neg.ctm
94
95 ```ctm
96 topic
97 occtype: -.3.
98 ```
99
100 Original final LF present.
101
102 ### ctm/in/occurrence-decimal-leading-zeros.ctm
103
104 ```ctm
105 topic
106 occtype: 0000000000000.3.
107 ```
108
109 Original final LF absent.
110
111 ### ctm/in/occurrence-decimal-minus-sign.ctm
112
113 ```ctm
114 topic
115 occtype: -9.3.
116 ```
117
118 Original final LF absent.
119
120 ### ctm/in/occurrence-decimal-plus-sign.ctm
121
122 ```ctm
123 topic
124 occtype: +9.3.
125 ```
126
127 Original final LF absent.
128
129 ### ctm/in/occurrence-decimal-1.0.ctm
130
131 ```ctm
132
133 topic
134 occtype: 1.0.
135 ```
136
137 Original final LF absent.
138
139 ## Independent semantic controls
140
141 Bare `01.00`, `+1.0` and `1.000` already share the stored string `"1.0"`
142 under the reader policy and merge when the other properties match.
143 Qualified `"01.00"^^xsd:decimal` and `"1.0"^^xsd:decimal` retain different
144 strings and remain two Occurrences even though both serialize as `1.0`.
145 Two separate maps with those strings can have the same XML while retaining
146 unequal stored values. Canonical XML is not used to define TMDM equality.
147
148 Qualified `"1"^^xsd:decimal` and `"1"^^xsd:integer` retain the exact same string
149 but remain different Occurrences solely because the Datatype Locators differ.
150 They output `1.0` and `1`. Stored strings `+002.00`, `-10.00`, `001.00`, `2.0`
151 sort in that order; output values are `2.0`, `-10.0`, `1.0`, `2.0`, demonstrating
152 why sorting formatted output or mathematical values would be wrong.
153
154 The suite checks all permutations of representative sets, lexical duplicates
155 and Item Identifier union after Parent/Type merges, raw stored canonical
156 references, retained replacement handles, malformed inputs and source positions,
157 negative zero, 10,000-digit integer/fraction components and tiny values with
158 10,000 fractional leading zeroes. Serializing never rewrites the model or its
159 retained handles. An independent Python Decimal arithmetic oracle checks 157
160 implicit/explicit probes with sufficient explicit precision and no binary float.
161
162 ## API and executable inspection
163
164 Existing deserialization, model and CXTM APIs are unchanged. New pure helpers
165 in `topic-maps.tmdm` are `decimal-datatype`, `decimal-lexical-p` and
166 `canonical-decimal-lexical-form` (returns NIL for invalid forms). The model API
167 validates supported datatype lexical forms before mutation, retaining the string.
168 Malformed qualified values signal `CTM-REJECTION :INVALID-DECIMAL-VALUE` at the
169 string token's source position; model insertion uses
170 `OCCURRENCE-ERROR :INVALID-DECIMAL-VALUE`, and a private malformed item is refused
171 by `CXTM-ERROR :INVALID-DECIMAL-VALUE`. Undefined prefixes and unsupported
172 features remain separate conditions. The lexer retains raw decimal spelling.
173
174 ```lisp
175 (require :asdf)
176 (asdf:initialize-source-registry '(:source-registry :ignore-inherited-configuration))
177 (asdf:load-asd #p"/Users/rgb/workspace/topic-maps/topic-maps.asd")
178 (asdf:load-system "topic-maps/ctm")
179 (asdf:load-system "topic-maps/cxtm")
180 (let* ((doc "https://example.invalid/decimals.ctm")
181 (map (topic-maps.ctm:deserialize-string
182 "%prefix xsd <http://www.w3.org/2001/XMLSchema#> topic occtype: +001.2300; occtype: \"001.2300\"^^xsd:decimal."
183 :document-iri doc))
184 (owner (first (topic-maps.tmdm:topic-map-topics map)))
185 (items (topic-maps.tmdm:topic-occurrences owner)))
186 (assert (= 2 (length items)))
187 (assert (equal '("1.23" "001.2300") (mapcar #'topic-maps.tmdm:occurrence-value items)))
188 (assert (every (lambda (item)
189 (string= (topic-maps.tmdm:decimal-datatype)
190 (topic-maps.tmdm:occurrence-datatype item))) items))
191 (assert (eq (topic-maps.tmdm:occurrence-type (first items))
192 (topic-maps.tmdm:occurrence-type (second items))))
193 (describe (second items))
194 (describe (topic-maps.tmdm:occurrence-type (second items)))
195 (write-string (topic-maps.cxtm:canonicalize map :base-locator doc)))
196 ```
197
198 ## Remaining limits
199
200 Only the exact xsd:decimal locator receives Decimal treatment; no derived-type
201 coercion, general numeric implementation, exponent syntax, or floating-point
202 conversion is introduced. Explicit XSD decimal strings may omit a fractional
203 component; bare CTM decimals may not. `1.` retains the Slice-7 Integer behavior.
204 No application-defined precision limit is introduced beyond finite Lisp strings
205 and available memory. Date/DateTime, CTM `*` semantics, nonempty Scope,
206 Reification, Variants, Templates, cross-document merging and HyperDoc integration
207 remain unimplemented. Unselected imported cases stay NOT-RUN even where the
208 new subset might happen to parse them. No selected draft/published-standard
209 conflict was found.