]> dreyeck.freedombox.rocks Git - topic-maps.git/blob - README.md
Support CTM associations with arbitrarily many roles
[topic-maps.git] / README.md
1 # Common Lisp Topic Maps — Slice 17A
2
3 A working ANSI Common Lisp reader for CTM identity/name syntax and
4 Associations with one or more Roles, with same-map merging of Topics, Names, Associations
5 and Roles, plus string/IRI-valued Occurrences, in a bounded TMDM model. HyperDoc remains a possible future client
6 and is neither the codebase nor a dependency. Slice 4 adds a pure, independent
7 CXTM canonicalizer for this model. Templates and cross-document merging remain
8 outside the implementation. Slice 5 adds Occurrence items and replacement merging
9 without expanding Scope, Reification or Variants. Slice 6 adds document-local
10 prefixes, QName resolution and explicit datatype locators on quoted Occurrence
11 values. Unknown datatypes retain their lexical values; Date timezone CXTM rules
12 remain explicitly unsupported. Slice 7 adds integer literals, lexical validation
13 and pure xsd:integer canonical output; qualified strings retain their stored
14 lexical values. Slice 8 adds decimal literals, exact character-based xsd:decimal
15 canonicalization and lexical validation, with the same separation of stored
16 strings, TMDM equality and canonical serialization. Slice 9 adds timezone-free
17 xsd:date literals, Gregorian calendar validation and pure CXTM Date output.
18 Negative-year leap validation follows the documented XSD 1.0 E.1 interpretation.
19 Slice 10 adds xsd:dateTime with absent/zero zones, exact fractional precision and
20 midnight rollover. A noncandidate leap-second baseline disagrees with XSD D.1
21 following-minute interpretation and remains FAIL. Slice 11 adds exact nonzero-offset UTC canonicalization; the original timezone2
22 contract now executes and reports its known inconsistent baseline as FAIL.
23 Nonzero-offset BCE day rollover and leap-second contexts remain explicitly
24 unsupported; existing zero/absent-zone behavior is retained.
25 Slice 12A adds explicitly typed Topic Names. Slice 13 adds Scope Topic sets to
26 Names, Occurrences and Associations, including reference replacement, duplicate
27 merging and canonical output. Slice 14 adds unreified Variants with proper Scope
28 supersets, canonical merge handles and pure CXTM output. Slice 15 adds per-document
29 Wildcards and recursive Embedded Topics through the shared Topic reference reader.
30 Slice 16A adds general Item Identifiers for all seven existing information-item
31 kinds, including metadata-preserving replacement merges and pure CXTM output.
32 Slice 16B adds same-map Reification, full merge cascades, atomic semantic
33 preflight and all six CTM Reifier attachment positions.
34 Slice 17A removes the two-Role CTM limit, preserving the existing model and
35 canonicalizer. `isa`/`ako` and Core-Subject identity protection remain open for 17B.
36
37 Start with [Associations with Arbitrarily Many Roles](docs/reading-association-arity.md)
38 for the actual seven-Topic/three-Role example and executable inspection forms.
39 See [Slice 17A validation](docs/slice-17a-validation.md) for the full offline run
40 and the single new byte-exact original contract.
41
42 Start with [Reification](docs/reading-reification.md) for actual Reifier/Reified
43 objects, normative clauses, cascade handling and inspection forms.
44 See [General Item Identifiers](docs/reading-item-identifiers.md) for the
45 uniform API, retained handles, exact boundaries and executable inspection forms.
46
47 Systems: `topic-maps` (the retained foundation), `topic-maps/tests`,
48 `topic-maps/tmdm`, `topic-maps/ctm`, `topic-maps/ctm/tests` and
49 `topic-maps/ctm/merging-tests` and `topic-maps/ctm/association-tests`, plus `topic-maps/cxtm` and
50 `topic-maps/cxtm/tests`, `topic-maps/ctm/occurrence-tests` and `topic-maps/ctm/prefix-tests` and `topic-maps/ctm/integer-tests`, plus `topic-maps/ctm/decimal-tests` and `topic-maps/ctm/date-tests` and `topic-maps/ctm/datetime-tests` and `topic-maps/ctm/timezone-tests`, `topic-maps/ctm/typed-name-tests` and `topic-maps/ctm/scope-tests`, plus `topic-maps/ctm/variant-tests` and `topic-maps/ctm/reference-tests`.
51 The core reader/model uses ANSI Common Lisp; the foundation/tooling uses
52 ASDF/UIOP and Python's standard library. No Quicklisp packages are required.
53 `topic-maps/tmdm/identifier-tests` and `topic-maps/tmdm/reification-tests` add the
54 general identifier and Reification regression suites.
55 `topic-maps/ctm/association-arity-tests` adds n-ary syntax, merge/handle checks
56 and the original `association-duplicate-role.ctm` comparison.
57
58 Start with [Reading Topic References](docs/reading-topic-references.md) for Wildcard
59 IRIs/counters, recursive Embedded Topics and actual object relationships.
60 [Reading Variants](docs/reading-variants.md) for Parent Name relationships,
61 explicit/resulting Scope, merge preflight and the four original contracts.
62 [Reading Scope](docs/reading-scope.md) for Scope APIs, actual Topic
63 references, merge handles and the three original comparisons.
64 [Reading Typed Topic Names](docs/reading-typed-topic-names.md) covers Slice 12A.
65 [Reading DateTime Timezone Canonicalization](docs/reading-datetime-timezones.md)
66 for UTC conversion, calendar limits, actual stored strings and executable inspection.
67 [Reading DateTime Occurrences](docs/reading-datetime-occurrences.md)
68 for lexical/value/canonical distinctions and the two normative discrepancies.
69 [Reading Date Occurrences](docs/reading-date-occurrences.md)
70 for the ten original Date cases, calendar rules, stored values and inspection.
71 [Reading Decimal Occurrences](docs/reading-decimal-occurrences.md)
72 for the seven Decimal cases, string representation, equality and inspection.
73 [Reading Integer Occurrences](docs/reading-integer-occurrences.md)
74 for integer representation, lexical equality, canonicalization and inspection.
75 [Reading Prefixes and Explicit Datatypes](docs/reading-prefixes-and-datatypes.md)
76 for the seven original cases, current API, object properties and diagnostics.
77 [Reading the First Four CTM Occurrences](docs/reading-first-occurrences.md)
78 for actual parent/type references, literal values, datatypes and executable inspection.
79 [Reading Canonical Topic Maps](docs/reading-canonical-topic-maps.md)
80 for the CXTM API, normative clauses, base-locator policy and actual byte comparisons.
81 [Reading the First CTM Associations](docs/reading-first-ctm-associations.md)
82 for actual Association/Role items, late Topic merges and current subset boundaries.
83 [Reading Topic Identity and Merging](docs/reading-topic-merging.md)
84 for the current API, reference replacement, identifier tails and exact subset
85 boundaries. See [Reading the First Five CTM Documents](docs/reading-first-five-ctm-documents.md)
86 for original sources, actual objects, normative clauses, baseline evidence and
87 copy-pasteable DESCRIBE forms. [API and boundaries](docs/slice-1-api-and-boundaries.md)
88 document the accepted Slice 1 baseline; the Slice 2 document supersedes its
89 statements about absent identity tails, merging and immutable Name references.
90 The Slice 3 document supersedes historical statements about absent Associations/Roles.
91
92 The pinned suite targets the **CTM draft dated 2010-03-31**. Passing it alone
93 does not establish full ISO/IEC 13250-6:2010 conformity.
94
95 ## Load and inspect
96
97 In a new Common Lisp process, with this checkout selected:
98
99 ```lisp
100 (require :asdf)
101 (asdf:initialize-source-registry
102 '(:source-registry :ignore-inherited-configuration))
103 (asdf:load-asd #p"/Users/rgb/workspace/topic-maps/topic-maps.asd")
104 (asdf:load-system "topic-maps/ctm")
105 (asdf:load-system "topic-maps")
106 (defparameter *corpus* (topic-maps.test-foundation:discover-corpus))
107 (topic-maps.test-foundation:summarize-corpus *corpus*)
108 (topic-maps.test-foundation:select-cases *corpus* :category :invalid)
109 (topic-maps.test-foundation:select-cases *corpus* :id "ctm/in/simple.ctm")
110 (asdf:test-system "topic-maps/tests")
111 (asdf:test-system "topic-maps/ctm/tests")
112 (asdf:test-system "topic-maps/ctm/merging-tests")
113 (asdf:test-system "topic-maps/ctm/association-tests")
114 (asdf:test-system "topic-maps/ctm/decimal-tests") ; includes retained Integer/Prefix/Occurrence/CXTM suites
115 (topic-maps.test-foundation:write-validation-report
116 *corpus* #p"/tmp/topic-maps-inventory.json")
117 ```
118
119 Change only the checkout pathname when using another machine. The case's
120 Document IRI remains independent of that pathname. Records are inspectable
121 Common Lisp structures, with actual source/baseline pathnames and explicit
122 contracts, results, statuses, diagnostics and reviewed clauses.
123
124 ## Offline verification
125
126 ```sh
127 # SBCL must already be installed. No network is used by this command.
128 TOPIC_MAPS_REPORT=/tmp/topic-maps-validation.json ./scripts/test.sh
129 ```
130
131 Set `SBCL=/absolute/path/to/sbcl` if needed. The launcher starts a new process,
132 ignores system/user initialization, clears inherited ASDF source configuration
133 and writes FASLs outside the repository. It runs byte-integrity fixtures,
134 Common Lisp infrastructure checks and an independent JSON-schema/content check
135 when a report pathname is supplied. `nix develop path:.` supplies a minimal,
136 pinned SBCL/Python/Git environment; initial environment provisioning may need
137 network access, while test execution after provisioning does not.
138
139 The complete upstream corpus is already vendored, unchanged. If absent in a
140 source distribution, acquire it once:
141
142 ```sh
143 python3 scripts/corpus.py acquire
144 python3 scripts/corpus.py verify
145 ```
146
147 Acquisition uses the exact commit and archive digest in
148 [corpus-lock.json](references/corpus-lock.json), refuses to merge over existing
149 contents and verifies every file. Verification is offline. Valid CTM inputs
150 are selected only under `ctm/in`, invalid ones only under `ctm/invalid`, and
151 CXTM baselines only under `ctm/baseline`. Auxiliary files are reported separately.
152
153 ## Current validation and historical foundation
154
155 The complete pinned corpus remains unchanged: 231 valid CTM inputs, 42 invalid
156 inputs, 231 CXTM baselines and 22 CTM auxiliary files. Ninety-five selected positive
157 cases pass actual CXTM byte comparisons. Twelve specific rejection contracts pass.
158 Two executed DateTime contracts remain FAIL because of documented normative
159 baseline discrepancies; 164 contracts remain NOT-RUN. A green regression run
160 preserves those two FAILs and never changes an upstream baseline. Unsupported
161 constructs and I/O errors cannot count as the required CTM rejection.
162
163 The full launcher runs the retained 54 foundation checks, two Python integrity
164 fixtures, 123 retained identity checks, 646 merge checks, 657 Association checks,
165 imported-report validation and selected baseline
166 property evidence, 445 retained CXTM checks, 1,101 Occurrence checks, 150 prefix/datatype checks, 665 integer checks, 715 Decimal checks, 1,806 Date checks, 217 DateTime checks, 1,944 timezone checks, 106 typed-name checks, 293 Scope checks, 324 Variant checks and 238 Topic Reference checks, plus
167 244 general Item Identifier checks, 236 Reification checks, 337 n-ary Association
168 checks and 16,294 independent Unicode NFC oracle
169 comparisons. The oracle needs Python UCD 15.0 or 15.1 (tables use UCD 15.1.0).
170 A report path also produces `<path>.semantics.json` with
171 separate model observations, plus `<path>.merging.json`, `<path>.associations.json`
172 and `<path>.cxtm.json` (complete actual/expected XML and byte counts),
173 plus `<path>.occurrences.json` with separate actual Occurrence objects/properties
174 and `<path>.prefixes.json` with all seven selected observations, plus `<path>.integers.json` for integer objects
175 and independently checked arithmetic probes, plus `<path>.decimals.json` with
176 seven Decimal object observations and 157 independent Decimal arithmetic probes,
177 plus `<path>.dates.json` with five original Date results, five original rejections,
178 396 independent date/calendar probes and 15 century decisions. `<path>.datetimes.json` records five actual DateTime
179 models, 45 independent lexical/canonical probes and two executed baseline
180 discrepancies. `<path>.timezones.json` adds exact UTC, fraction and model-purity
181 probes, including every one of the 1681 permitted signed offset minutes.
182 The merge checks include 128 graph/order variants with an independent reachability
183 oracle. Earlier slices recorded version/relocation comparisons; Slice 17A uses
184 the existing single-runtime environment. `<path>.typed-names.json` records Name value/Parent/Type relations;
185 `<path>.scopes.json` records Scope object relations and an ordering witness.
186 `<path>.variants.json` records Variant/Parent/Scope observations and XML ordering.
187 `<path>.references.json` records the new source/counter/model observations.
188 `<path>.identifiers.json` and `<path>.reification.json` retain their model evidence;
189 `<path>.arity.json` adds actual Role references and twelve canonical XML witnesses.
190 See [Slice 17A validation](docs/slice-17a-validation.md) for current evidence.
191 [Slice 12A validation](docs/slice-12a-validation.md) records explicit Name types.
192 [Slice 11 validation](docs/slice-11-validation.md) preserves the prior timezone evidence.
193 [Slice 10 validation](docs/slice-10-validation.md) preserves the earlier DateTime baseline.
194 [Slice 9 validation](docs/slice-9-validation.md) records the accepted Date subset.
195 [Slice 8 validation](docs/slice-8-validation.md) records the accepted Decimal subset.
196 [Slice 7 validation](docs/slice-7-validation.md) records the accepted Integer subset.
197 [Slice 6 validation](docs/slice-6-validation.md) records the preceding accepted subset.
198 [Slice 5 validation](docs/slice-5-validation.md) records the preceding accepted subset.
199 [Slice 4 validation](docs/slice-4-validation.md) records the accepted baseline.
200 [Slice 3 validation](docs/slice-3-validation.md),
201 [imported contracts](reports/slice-1/validation.json) and
202 [semantic observations](reports/slice-1/semantics.json).
203
204 [Slice 0 inventory](docs/inventory.md), [architecture](docs/architecture.md),
205 [coverage](docs/coverage.md) and [validation](docs/validation.md) remain historical
206 foundation records. Their original report/logs are preserved. Their statements
207 about absent reader/model systems describe Slice 0, not the current branch.
208 Slice 1 is built from `9c83db34eb1ba8091b004433f088056717dbaa7e` on
209 `codex/ctm-slice-1`. Slice 2 starts from the accepted Slice 1a commit
210 `fb60afd548d679d83fd8ec76bc5ff384c1e23e5f` on `codex/ctm-slice-2`.
211 Slice 13 starts from `788899aea1a20b9cd8237c6106e6613e180d7362` on
212 `codex/ctm-slice-13`.
213 Slice 14 starts from `aeb017e4f4846f2e6a701e581d20eef84a8b83d9` on
214 `codex/ctm-slice-14`.
215 Slice 11 starts from `8741d155d948c7317945eee423a977940cad995e` on
216 `codex/ctm-slice-11`.
217 Slice 10 starts from `2ad41cd1a945a4ce97824464de40646bb899259e` on
218 `codex/ctm-slice-10`.
219 Slice 9 starts from `195d5f5444d776096b765d04c815c23c3db5f84c` on
220 `codex/ctm-slice-9`.
221 Slice 8 starts from `17dd58775e25ab495e3c83deb4b5a36f5e5a124b` on
222 `codex/ctm-slice-8`.
223 Slice 7 starts from `3924d1eac5a070fe5a6a26683e23ac338793f544` on
224 `codex/ctm-slice-7`.
225 Slice 6 starts from `437ad53ea7539d36db3fa871d0931650b7a6a03e` on
226 `codex/ctm-slice-6`.
227 Slice 5 starts from `8036a2b1ce009218f33d300edcc5f715ec1d1124` on
228 `codex/ctm-slice-5`.
229 Slice 4 starts from `35648c4dcc71e8dc6448b64abcacb438e53d64a2` on
230 `codex/ctm-slice-4`.
231 Slice 3 starts from accepted `66664dbff6275d60dab1bc6a232fa70f7b916b8c` on
232 `codex/ctm-slice-3`. Await review before continuing to the next slice.
233
234 Slice 15 starts from `4f7b4d26d40f8f5566594aa2e3ad6296d44fc4c3` on `codex/ctm-slice-15`.