]> dreyeck.freedombox.rocks Git - topic-maps.git/blob - docs/reading-ctm-include.md
docs(ctm): preserve Template Tail grammar investigation and evidence
[topic-maps.git] / docs / reading-ctm-include.md
1 # Reading CTM Include and Imported Templates
2
3 Slice 19C builds on `5e716c078b885e97ebe858230ce7d8578e65ae4b` and the unchanged
4 corpus `cd165b150176c1d6842b37be9cc9051b40d4996b` (draft 2010-03-31).
5 Published CTM remains authoritative. This slice implements Include; it does not
6 claim complete CTM/TMDM conformance or resolve the two original baseline conflicts.
7
8 ## Requirements and chosen execution contract
9
10 [CTM 3.2, 3.3.6, 3.3.8, 3.14–3.16.2](https://www.isotopicmaps.org/ctm/ctm.html)
11 define the environments, identifier construction, Templates and Include.
12 Include processes a child immediately, using its resource IRI as Document IRI,
13 the immediate parent's **Document IRI** as child Wildcard IRI, and the current
14 Counter. Each level adds aliases for existing child Topic IIs starting with
15 that child's Document IRI; the suffix is resolved against the parent's Document
16 IRI. Existing identifiers are retained. The final child Counter is returned.
17 Templates are imported as declarations at the inclusion point; Prefix bindings
18 and named-Wildcard caches are not imported.
19
20 Body identifiers, relative IRIs and QNames in imported Templates execute in the
21 caller's active Document-IRI/Prefix environment. This is the explicit, reviewed
22 interpretation for this slice, not a lexical closure over the definition's
23 Prefix bindings. Definition-source identity and Document IRI are retained as
24 provenance. Arguments use the existing contextual binding rules; actual Topics
25 are canonical handles, not generated identity substitutes. Forward lookup uses
26 the current Template registry. Each invocation has fresh named-Wildcard bindings
27 and shares the active Counter.
28
29 [TMDM 4.3, 5.1–5.8 and 6.1–6.7](https://www.isotopicmaps.org/sam/sam-model/2006-06-18/)
30 remain the identity/merge authority. Alias addition and Map content transfer
31 use the existing operations, including Scope, Variant, Reifier and Core Subject
32 invariants. Source Map identifiers/reifier are not transferred as root metadata.
33 [CXTM 3.13, 3.17–3.18 and 4](https://www.isotopicmaps.org/cxtm/cxtm.html)
34 preserve all identifier entries, with canonical locator representation/order.
35 No serializer rule or TMDM representation is changed in this slice.
36
37 ## Public API and resource boundary
38
39 `deserialize-string` and `deserialize-file` keep their existing keyword interface
40 and exactly two return values: Map and final Counter. `:resource-resolver` is a
41 function called with the absolute resource IRI and these keywords:
42
43 ```text
44 (:purpose :include :from-document-iri parent-document-iri :notation nil)
45 ```
46
47 It returns an octet vector and optional string source key. Only the resource
48 IRI supplies the child's identity base; the source key is diagnostic. Existing
49 strict UTF-8/BOM decoding is reused. No implicit filesystem or network fallback
50 exists. Invalid CTM bytes/source are rejected, not accepted as arbitrary data.
51
52 Missing resources, resolver errors and malformed resolver results are resource
53 conditions. Missing resolver, active cycles and configured depth limits are
54 explicit unsupported/resource-policy conditions. Cycle/depth rejection is an
55 implementation policy, not a normative CTM grammar prohibition. Existing
56 Mergemap behavior remains independent: no Include aliasing, Template import or
57 Counter propagation is added to Mergemap.
58
59 Alias inputs are snapshotted once per Include level. Matching is a literal
60 string prefix, not a fragment-boundary test. Suffixes are already locator
61 strings and are **not percent-decoded a second time**. Only Topic IIs receive
62 aliases: SIs, SLs, Occurrence values and root identifiers do not.
63
64 Template-name/arity conflicts are checked before caller mutation. Alias work
65 uses the private child graph. Existing `import-map-content` preflight protects
66 the caller Map on semantic conflicts. Counter and Template-registry transfer
67 follow successful model transfer. This is a single-threaded semantic rejection
68 contract; it is not a new general transaction system.
69
70 ## Source preservation
71
72 Definitions retain their name/arity, original tokens, original source text/key,
73 Document IRI and definition position. Template execution changes lexical
74 source/line context to the definition while keeping the caller's semantic
75 environment. Error frames separately retain definition source/position/Document
76 IRI and invocation source/position/active Document IRI. Nested invocations retain
77 the complete chain. No source text substitution or Lisp evaluation occurs.
78
79 ## Two original baseline discrepancies
80
81 Let `P` be the fixed corpus Document-IRI prefix ending in `/ctm/in/`.
82
83 For `include-template.ctm`, `T=P+include-template.ctm`, `S=P+include-template.sub`.
84 The child contains only a definition and creates zero Topics. Its Include alias
85 pass therefore has no operands. Later `template(topic)` creates `{T#topic}` and
86 Type `{T#occtype}` with one string Occurrence `"Test"`. No rule in the reviewed
87 clauses supplies a future `S#occtype` alias. Actual CXTM: **353 bytes**; original:
88 **401 bytes**, with extra `include-template.sub#occtype`. Topic numbering also
89 changes. The registered original comparison is **FAIL**; model assertions pass.
90
91 For `include-wildcard2.ctm`, use A for the main source, B for `.sub`, C for `.sub2`:
92
93 | Processor | Document IRI | Wildcard IRI | Entry → exit Counter |
94 |---|---|---|---|
95 | C | C | B | 0 → 1 |
96 | B | B | A | 0 → 2 |
97 | A | A | A | 0 → 3 |
98
99 C creates `B#$__1.foo`. C→B adds no alias because that II does not start with C.
100 B creates `A#$__2.foo`. B→A adds `A#$__1.foo` while retaining `B#$__1.foo`.
101 A creates `A#$__3.foo`. The final three named subjects have these II sets:
102
103 - Sub Sub: `{A#$__1.foo, B#$__1.foo}`
104 - Sub: `{A#$__2.foo}`
105 - Master: `{A#$__3.foo}`
106
107 All three Names share the same actual default Type Topic. Actual CXTM:
108 **719 bytes**; original: **669 bytes**, omitting `include-wildcard2.sub#$__1.foo`.
109 Canonical Topic order changes because one identifier set is larger. This is a
110 missing baseline identifier, not merely a relative-locator spelling difference.
111 The registered comparison remains **FAIL**. The root Wildcard IRI is not silently
112 substituted for the immediate parent's Document IRI.
113
114 These findings distinguish published-rule interpretation, upstream baseline
115 evidence and locally executed observations. No upstream input/baseline is edited.
116
117 ## Inspect actual objects
118
119 Run from the repository root in a fresh SBCL process. The resolver below supplies
120 only the one known fixture; it is not a general filesystem/network loader.
121
122 ```lisp
123 (require :asdf)
124 (asdf:initialize-source-registry '(:source-registry :ignore-inherited-configuration))
125 (asdf:load-asd (truename "topic-maps.asd"))
126 (asdf:load-system "topic-maps/ctm")
127 (asdf:load-system "topic-maps/cxtm")
128 (let* ((base "https://ctm-tests.example.invalid/cd165b150176c1d6842b37be9cc9051b40d4996b/ctm/in/")
129 (doc (concatenate 'string base "include-template.ctm"))
130 (child (concatenate 'string base "include-template.sub"))
131 (root (asdf:system-source-directory "topic-maps"))
132 (resolver
133 (lambda (iri &key purpose from-document-iri notation)
134 (assert (and (eq purpose :include) (null notation) (string= from-document-iri doc)))
135 (when (string= iri child)
136 (with-open-file (in (merge-pathnames "vendor/cxtm/ctm/in/include-template.sub" root)
137 :element-type '(unsigned-byte 8))
138 (let ((bytes (make-array (file-length in) :element-type '(unsigned-byte 8))))
139 (read-sequence bytes in) (values bytes "ctm/in/include-template.sub")))))))
140 (multiple-value-bind (map counter)
141 (topic-maps.ctm:deserialize-file
142 (merge-pathnames "vendor/cxtm/ctm/in/include-template.ctm" root)
143 :document-iri doc :source-name "ctm/in/include-template.ctm" :resource-resolver resolver)
144 (let* ((owner (find-if (lambda (topic) (member (concatenate 'string doc "#topic")
145 (topic-maps.tmdm:item-identifiers topic) :test #'string=))
146 (topic-maps.tmdm:topic-map-topics map)))
147 (occurrence (first (topic-maps.tmdm:topic-occurrences owner))))
148 (assert (= counter 0))
149 (assert (eq owner (topic-maps.tmdm:occurrence-parent occurrence)))
150 (assert (equal (list (concatenate 'string doc "#occtype"))
151 (topic-maps.tmdm:item-identifiers (topic-maps.tmdm:occurrence-type occurrence))))
152 (describe occurrence)
153 (describe (topic-maps.tmdm:occurrence-type occurrence))
154 (write-string (topic-maps.cxtm:canonicalize map :base-locator doc)))))
155 ```
156
157 Use `reports/slice-19c/include.json` for actual original-source model observations,
158 resource requests, processor environments and every before/after alias pass.
159 Use `cxtm.json` and `actual/` for actual canonical outputs and unchanged expected
160 bytes. Model checks passing do not turn a mismatching original contract into PASS.