rickub
nandi/clonimpublic Fork
⑂ ee111f4
Branches
main
Tags
v0.1.0 v0.1.1 v0.1.10 v0.1.11 v0.1.12 v0.1.13 v0.1.14 v0.1.15 v0.1.16 v0.1.17 v0.1.2 v0.1.3 v0.1.4 v0.1.5 v0.1.6 v0.1.7 v0.1.8 v0.1.9 View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/clonim.git
git clone ssh://git@rickub.com/nandi/clonim.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Download ZIP

Implement static namespaces and explicit stdlib imports; fix diagnostics

nandithebull committed 2026-09-20T20:48:03-07:00 Browse files
ee111f4 parent: d48248e
added .clj-kondo/config.edn +10 -0
new file mode 100644
@@ -0,0 +1,10 @@
1+{:config-in-ns
2+ {clonim.core {:linters {:unresolved-symbol {:exclude [*epoch-time-ms*]}}}
3+ ;; These regression fixtures intentionally redefine vars and shadow core.
4+ directcalls {:linters {:redefined-var {:level :off}}}
5+ rebinding {:linters {:redefined-var {:level :off}}}
6+ intrinsics {:linters {:redefined-var {:level :off}}}
7+ ;; Explicit do forms test evaluation order; fusion also rebinds core map.
8+ fusion {:linters {:redundant-do {:level :off}
9+ :redefined-var {:level :off}}}
10+ evalorder {:linters {:redundant-do {:level :off}}}}}
new file mode 100644
@@ -0,0 +1,10 @@
1+{:config-in-ns
2+ {clonim.core {:linters {:unresolved-symbol {:exclude [*epoch-time-ms*]}}}
3+ ;; These regression fixtures intentionally redefine vars and shadow core.
4+ directcalls {:linters {:redefined-var {:level :off}}}
5+ rebinding {:linters {:redefined-var {:level :off}}}
6+ intrinsics {:linters {:redefined-var {:level :off}}}
7+ ;; Explicit do forms test evaluation order; fusion also rebinds core map.
8+ fusion {:linters {:redundant-do {:level :off}
9+ :redefined-var {:level :off}}}
10+ evalorder {:linters {:redundant-do {:level :off}}}}}
added .lsp/config.edn +1 -0
new file mode 100644
@@ -0,0 +1 @@
1+{:source-paths ["stdlib" "examples"]}
new file mode 100644
@@ -0,0 +1 @@
1+{:source-paths ["stdlib" "examples"]}
modified README.md +37 -2
@@ -23,6 +23,39 @@ and flags slowdowns above 10% versus the latest compatible run. Requires Jolt
2323 and a clean working tree. Use `just bench-store 15` to change the threshold;
2424 see [benchmark history](bench/README.md) for metrics and exit codes.
2525
26+Source-level libraries are loaded explicitly. Use
27+`(require '[clonim.core :refer [now-ms]])` to load `stdlib/clonim/core.clj`. Host-dependent operations remain small runtime primitives; for
28+example, the stdlib `now-ms` function wraps the `*epoch-time-ms*` primitive.
29+
30+## Namespaces and libraries
31+
32+Each file may start with a namespace declaration:
33+
34+```clojure
35+(ns demo.main
36+ (:require [clonim.core :as clock]))
37+
38+(println (clock/now-ms))
39+```
40+
41+Top-level `(require 'clonim.core)` loads the library without importing names;
42+use `clonim.core/now-ms`, an `:as` alias, or an explicit `:refer [now-ms]`.
43+Definitions belong to their namespace (`user` for scripts without `ns`).
44+Locals shadow vars, and normal core functions resolve to `clojure.core`.
45+Forward references require `declare`. A later `def +` creates a var in the
46+current namespace rather than changing already-resolved core references.
47+
48+Libraries are loaded once, before their dependents. Namespace names map to paths
49+with dots as separators and hyphens as underscores. Add search roots with the
50+repeatable `--source-path <directory>` CLI option; these precede the working
51+directory, input file's directory, and bundled `stdlib/`. Libraries are never
52+loaded implicitly merely because their source root is available.
53+
54+This is a static subset: one leading `ns` per file, literal top-level `require`,
55+`:as`, explicit `:refer`, and `defn-` privacy. Dynamic namespace operations,
56+reload options, `:refer :all`, and arbitrary macro expansion are unsupported.
57+Missing dependencies, missing vars, and dependency cycles produce errors.
58+
2659 ## Pipeline
2760
2861 | stage | file | what it does |
@@ -31,6 +64,7 @@ see [benchmark history](bench/README.md) for metrics and exit codes.
3164 | analyzer + codegen | `src/compiler.nim` | expands the macro set to core special forms, emits Nim statements |
3265 | runtime | `src/runtime.nim` | the `Value` tagged union, the persistent vector/map, equality, printing, var cells, `call` |
3366 | core | `src/core.nim` | ~140 `clojure.core` builtins as Nim closures |
67+| stdlib | `stdlib/clonim/core.clj` | source-level helpers loaded by explicit require |
3468 | driver | `src/clonim.nim` | shells out to `nim c`, times each phase |
3569
3670 Codegen is statement-oriented: every form is compiled as "emit statements that
@@ -59,7 +93,7 @@ Maps and sets also keep insertion order: each entry carries an `ord` stamp and
5993 iteration sorts by it, so printing, `keys` and `vals` are deterministic the way
6094 Clojure's small array-maps are, without giving up hashed lookup.
6195
62-`examples/persistent-bench.clj` (`just bench`), n = 8000, `-d:release`:
96+`examples/persistent_bench.clj` (`just bench`), n = 8000, `-d:release`:
6397
6498 | operation | copy-on-write `seq` | trie / HAMT |
6599 |---|---:|---:|
@@ -127,11 +161,12 @@ library above returns them where Clojure does.
127161 chunking is the fix, at the cost of exact demand.
128162 - **Destructuring in parameter lists.** `(let [[a b] xs] …)` works; `(defn f
129163 [[a b]] …)` does not.
130-- Protocols/records, namespaces (`ns` is parsed and ignored), refs/agents,
164+- Protocols/records, dynamic namespace operations, refs/agents,
131165 `#()` literals, syntax-quote, transducers, Nim interop.
132166
133167 ## Tests
134168
135169 ```bash
136170 ./run-tests.sh # builds the compiler, diffs every example against tests/*.expected
171+nim r --path:src tests/test_namespaces.nim # namespace resolution and loader tests
137172 ```
@@ -23,6 +23,39 @@ and flags slowdowns above 10% versus the latest compatible run. Requires Jolt
23 and a clean working tree. Use `just bench-store 15` to change the threshold;23 and a clean working tree. Use `just bench-store 15` to change the threshold;
24 see [benchmark history](bench/README.md) for metrics and exit codes.24 see [benchmark history](bench/README.md) for metrics and exit codes.
25 25
26+Source-level libraries are loaded explicitly. Use
27+`(require '[clonim.core :refer [now-ms]])` to load `stdlib/clonim/core.clj`. Host-dependent operations remain small runtime primitives; for
28+example, the stdlib `now-ms` function wraps the `*epoch-time-ms*` primitive.
29+
30+## Namespaces and libraries
31+
32+Each file may start with a namespace declaration:
33+
34+```clojure
35+(ns demo.main
36+ (:require [clonim.core :as clock]))
37+
38+(println (clock/now-ms))
39+```
40+
41+Top-level `(require 'clonim.core)` loads the library without importing names;
42+use `clonim.core/now-ms`, an `:as` alias, or an explicit `:refer [now-ms]`.
43+Definitions belong to their namespace (`user` for scripts without `ns`).
44+Locals shadow vars, and normal core functions resolve to `clojure.core`.
45+Forward references require `declare`. A later `def +` creates a var in the
46+current namespace rather than changing already-resolved core references.
47+
48+Libraries are loaded once, before their dependents. Namespace names map to paths
49+with dots as separators and hyphens as underscores. Add search roots with the
50+repeatable `--source-path <directory>` CLI option; these precede the working
51+directory, input file's directory, and bundled `stdlib/`. Libraries are never
52+loaded implicitly merely because their source root is available.
53+
54+This is a static subset: one leading `ns` per file, literal top-level `require`,
55+`:as`, explicit `:refer`, and `defn-` privacy. Dynamic namespace operations,
56+reload options, `:refer :all`, and arbitrary macro expansion are unsupported.
57+Missing dependencies, missing vars, and dependency cycles produce errors.
58+
26 ## Pipeline59 ## Pipeline
27 60
28 | stage | file | what it does |61 | stage | file | what it does |
@@ -31,6 +64,7 @@ see [benchmark history](bench/README.md) for metrics and exit codes.
31 | analyzer + codegen | `src/compiler.nim` | expands the macro set to core special forms, emits Nim statements |64 | analyzer + codegen | `src/compiler.nim` | expands the macro set to core special forms, emits Nim statements |
32 | runtime | `src/runtime.nim` | the `Value` tagged union, the persistent vector/map, equality, printing, var cells, `call` |65 | runtime | `src/runtime.nim` | the `Value` tagged union, the persistent vector/map, equality, printing, var cells, `call` |
33 | core | `src/core.nim` | ~140 `clojure.core` builtins as Nim closures |66 | core | `src/core.nim` | ~140 `clojure.core` builtins as Nim closures |
67+| stdlib | `stdlib/clonim/core.clj` | source-level helpers loaded by explicit require |
34 | driver | `src/clonim.nim` | shells out to `nim c`, times each phase |68 | driver | `src/clonim.nim` | shells out to `nim c`, times each phase |
35 69
36 Codegen is statement-oriented: every form is compiled as "emit statements that70 Codegen is statement-oriented: every form is compiled as "emit statements that
@@ -59,7 +93,7 @@ Maps and sets also keep insertion order: each entry carries an `ord` stamp and
59 iteration sorts by it, so printing, `keys` and `vals` are deterministic the way93 iteration sorts by it, so printing, `keys` and `vals` are deterministic the way
60 Clojure's small array-maps are, without giving up hashed lookup.94 Clojure's small array-maps are, without giving up hashed lookup.
61 95
62-`examples/persistent-bench.clj` (`just bench`), n = 8000, `-d:release`:96+`examples/persistent_bench.clj` (`just bench`), n = 8000, `-d:release`:
63 97
64 | operation | copy-on-write `seq` | trie / HAMT |98 | operation | copy-on-write `seq` | trie / HAMT |
65 |---|---:|---:|99 |---|---:|---:|
@@ -127,11 +161,12 @@ library above returns them where Clojure does.
127 chunking is the fix, at the cost of exact demand.161 chunking is the fix, at the cost of exact demand.
128 - **Destructuring in parameter lists.** `(let [[a b] xs] …)` works; `(defn f162 - **Destructuring in parameter lists.** `(let [[a b] xs] …)` works; `(defn f
129 [[a b]] …)` does not.163 [[a b]] …)` does not.
130-- Protocols/records, namespaces (`ns` is parsed and ignored), refs/agents,164+- Protocols/records, dynamic namespace operations, refs/agents,
131 `#()` literals, syntax-quote, transducers, Nim interop.165 `#()` literals, syntax-quote, transducers, Nim interop.
132 166
133 ## Tests167 ## Tests
134 168
135 ```bash169 ```bash
136 ./run-tests.sh # builds the compiler, diffs every example against tests/*.expected170 ./run-tests.sh # builds the compiler, diffs every example against tests/*.expected
171+nim r --path:src tests/test_namespaces.nim # namespace resolution and loader tests
137 ```172 ```
modified bench/store.py +0 -2
@@ -2,12 +2,10 @@
22 """Record and commit a benchmark run; exit 1 after committing a regression."""
33
44 import argparse
5-
65 import hashlib
76 import json
87 import math
98 import os
10-
119 import platform
1210 import shutil
1311 import subprocess
@@ -2,12 +2,10 @@
2 """Record and commit a benchmark run; exit 1 after committing a regression."""2 """Record and commit a benchmark run; exit 1 after committing a regression."""
3 3
4 import argparse4 import argparse
5-
6 import hashlib5 import hashlib
7 import json6 import json
8 import math7 import math
9 import os8 import os
10-
11 import platform9 import platform
12 import shutil10 import shutil
13 import subprocess11 import subprocess
modified bench/test_measure.py +0 -1
@@ -2,7 +2,6 @@ import contextlib
22 import io
33 import json
44 import os
5-
65 import subprocess
76 import tempfile
87 import unittest
@@ -2,7 +2,6 @@ import contextlib
2 import io2 import io
3 import json3 import json
4 import os4 import os
5-
6 import subprocess5 import subprocess
7 import tempfile6 import tempfile
8 import unittest7 import unittest
modified bench/test_store.py +0 -1
@@ -3,7 +3,6 @@
33 import copy
44 import importlib.util
55 import json
6-
76 import subprocess
87 import tempfile
98 import unittest
@@ -3,7 +3,6 @@
3 import copy3 import copy
4 import importlib.util4 import importlib.util
5 import json5 import json
6-
7 import subprocess6 import subprocess
8 import tempfile7 import tempfile
9 import unittest8 import unittest
modified examples/bench.clj +3 -0
@@ -1,3 +1,6 @@
1+(ns bench
2+ (:require [clonim.core :refer [now-ms]]))
3+
14 (defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))
25 (let [t (now-ms)]
36 (println "fib 30 =" (fib 30) "in" (- (now-ms) t) "ms"))
@@ -1,3 +1,6 @@
1+(ns bench
2+ (:require [clonim.core :refer [now-ms]]))
3+
1 (defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))4 (defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))
2 (let [t (now-ms)]5 (let [t (now-ms)]
3 (println "fib 30 =" (fib 30) "in" (- (now-ms) t) "ms"))6 (println "fib 30 =" (fib 30) "in" (- (now-ms) t) "ms"))
modified examples/directcalls.clj +3 -0
@@ -1,3 +1,5 @@
1+(ns directcalls)
2+
13 (defn f [x] (* x 2))
24 (println (f 5))
35 (def f (fn [x] (+ x 100)))
@@ -15,6 +17,7 @@
1517 (defn fact [n] (if (< n 2) 1 (* n (fact (- n 1)))))
1618 (println (fact 10)) ; 3628800
1719
20+(declare od?)
1821 (defn ev? [n] (if (= n 0) true (od? (- n 1))))
1922 (defn od? [n] (if (= n 0) false (ev? (- n 1))))
2023 (println (ev? 10) (od? 7)) ; true true
@@ -1,3 +1,5 @@
1+(ns directcalls)
2+
1 (defn f [x] (* x 2))3 (defn f [x] (* x 2))
2 (println (f 5))4 (println (f 5))
3 (def f (fn [x] (+ x 100)))5 (def f (fn [x] (+ x 100)))
@@ -15,6 +17,7 @@
15 (defn fact [n] (if (< n 2) 1 (* n (fact (- n 1)))))17 (defn fact [n] (if (< n 2) 1 (* n (fact (- n 1)))))
16 (println (fact 10)) ; 362880018 (println (fact 10)) ; 3628800
17 19
20+(declare od?)
18 (defn ev? [n] (if (= n 0) true (od? (- n 1))))21 (defn ev? [n] (if (= n 0) true (od? (- n 1))))
19 (defn od? [n] (if (= n 0) false (ev? (- n 1))))22 (defn od? [n] (if (= n 0) false (ev? (- n 1))))
20 (println (ev? 10) (od? 7)) ; true true23 (println (ev? 10) (od? 7)) ; true true
modified examples/evalorder.clj +2 -0
@@ -1,3 +1,5 @@
1+(ns evalorder)
2+
13 (def log (atom []))
24 (defn note [x] (do (reset! log (conj (deref log) x)) x))
35
@@ -1,3 +1,5 @@
1+(ns evalorder)
2+
1 (def log (atom []))3 (def log (atom []))
2 (defn note [x] (do (reset! log (conj (deref log) x)) x))4 (defn note [x] (do (reset! log (conj (deref log) x)) x))
3 5
modified examples/fusion.clj +4 -2
@@ -1,3 +1,5 @@
1+(ns fusion)
2+
13 ;; results must match whether or not the pipeline fuses
24 (println (reduce + (map (fn [x] (* x x)) (filter even? (range 20))))) ; 1140
35 (println (reduce + 1000 (map inc (filter odd? (range 10))))) ; 1000+2+4+6+8+10=1030
@@ -29,8 +31,8 @@
2931 (println (deref log)) ; (:f :g :p :src)
3032
3133 ;; shadowing and rebinding must defeat fusion, not break it
32-(println (let [map (fn [f c] [99])] (reduce + (map inc [1 2 3])))) ; 99
34+(println (let [map (fn [_f _c] [99])] (reduce + (map inc [1 2 3])))) ; 99
3335
3436 ;; a def of one of the pipeline names defeats fusion for the whole program
35-(def map (fn [f c] [:replaced]))
37+(def map (fn [_f _c] [:replaced]))
3638 (println (reduce conj [] (map inc [1 2 3]))) ; [:replaced]
@@ -1,3 +1,5 @@
1+(ns fusion)
2+
1 ;; results must match whether or not the pipeline fuses3 ;; results must match whether or not the pipeline fuses
2 (println (reduce + (map (fn [x] (* x x)) (filter even? (range 20))))) ; 11404 (println (reduce + (map (fn [x] (* x x)) (filter even? (range 20))))) ; 1140
3 (println (reduce + 1000 (map inc (filter odd? (range 10))))) ; 1000+2+4+6+8+10=10305 (println (reduce + 1000 (map inc (filter odd? (range 10))))) ; 1000+2+4+6+8+10=1030
@@ -29,8 +31,8 @@
29 (println (deref log)) ; (:f :g :p :src)31 (println (deref log)) ; (:f :g :p :src)
30 32
31 ;; shadowing and rebinding must defeat fusion, not break it33 ;; shadowing and rebinding must defeat fusion, not break it
32-(println (let [map (fn [f c] [99])] (reduce + (map inc [1 2 3])))) ; 9934+(println (let [map (fn [_f _c] [99])] (reduce + (map inc [1 2 3])))) ; 99
33 35
34 ;; a def of one of the pipeline names defeats fusion for the whole program36 ;; a def of one of the pipeline names defeats fusion for the whole program
35-(def map (fn [f c] [:replaced]))37+(def map (fn [_f _c] [:replaced]))
36 (println (reduce conj [] (map inc [1 2 3]))) ; [:replaced]38 (println (reduce conj [] (map inc [1 2 3]))) ; [:replaced]
modified examples/hello.clj +2 -0
@@ -1,3 +1,5 @@
1+(ns hello)
2+
13 ;; a first taste
24 (defn greet [name]
35 (str "Hello, " name "!"))
@@ -1,3 +1,5 @@
1+(ns hello)
2+
1 ;; a first taste3 ;; a first taste
2 (defn greet [name]4 (defn greet [name]
3 (str "Hello, " name "!"))5 (str "Hello, " name "!"))
modified examples/intrinsics.clj +3 -1
@@ -1,3 +1,5 @@
1+(ns intrinsics)
2+
13 (println (+ 1 2) (- 5 3) (* 4 5) (inc 7) (dec 7)) ; 3 2 20 8 6
24 (println (+ 1.5 2) (* 2 0.5) (- 1 0.25) (inc 1.5)) ; 3.5 1.0 0.75 2.5
35 (println (< 1 2) (> 1 2) (<= 2 2) (>= 1 2)) ; true false true false
@@ -7,5 +9,5 @@
79 (println (+ 1 2 3) (< 1 2 3) (= 1 1 1)) ; non-binary arities: 6 true true
810 (println (let [+ (fn [a b] (* a b))] (+ 3 4))) ; local shadow: 12
911 (println (reduce + [1 2 3 4])) ; + as a value: 10
10-(def + (fn [a b] 999))
12+(def + (fn [_a _b] 999))
1113 (println (+ 1 2)) ; rebound: 999
@@ -1,3 +1,5 @@
1+(ns intrinsics)
2+
1 (println (+ 1 2) (- 5 3) (* 4 5) (inc 7) (dec 7)) ; 3 2 20 8 63 (println (+ 1 2) (- 5 3) (* 4 5) (inc 7) (dec 7)) ; 3 2 20 8 6
2 (println (+ 1.5 2) (* 2 0.5) (- 1 0.25) (inc 1.5)) ; 3.5 1.0 0.75 2.54 (println (+ 1.5 2) (* 2 0.5) (- 1 0.25) (inc 1.5)) ; 3.5 1.0 0.75 2.5
3 (println (< 1 2) (> 1 2) (<= 2 2) (>= 1 2)) ; true false true false5 (println (< 1 2) (> 1 2) (<= 2 2) (>= 1 2)) ; true false true false
@@ -7,5 +9,5 @@
7 (println (+ 1 2 3) (< 1 2 3) (= 1 1 1)) ; non-binary arities: 6 true true9 (println (+ 1 2 3) (< 1 2 3) (= 1 1 1)) ; non-binary arities: 6 true true
8 (println (let [+ (fn [a b] (* a b))] (+ 3 4))) ; local shadow: 1210 (println (let [+ (fn [a b] (* a b))] (+ 3 4))) ; local shadow: 12
9 (println (reduce + [1 2 3 4])) ; + as a value: 1011 (println (reduce + [1 2 3 4])) ; + as a value: 10
10-(def + (fn [a b] 999))12+(def + (fn [_a _b] 999))
11 (println (+ 1 2)) ; rebound: 99913 (println (+ 1 2)) ; rebound: 999
renamed examples/lazy_bench.clj +3 -0
similarity index 84%
rename from examples/lazy-bench.clj
rename to examples/lazy_bench.clj
@@ -1,3 +1,6 @@
1+(ns lazy-bench
2+ (:require [clonim.core :refer [now-ms]]))
3+
14 (def n 2000000)
25 (let [t0 (now-ms)
36 a (first (filter odd? (map inc (range n))))
similarity index 84%
rename from examples/lazy-bench.clj
rename to examples/lazy_bench.clj
@@ -1,3 +1,6 @@
1+(ns lazy-bench
2+ (:require [clonim.core :refer [now-ms]]))
3+
1 (def n 2000000)4 (def n 2000000)
2 (let [t0 (now-ms)5 (let [t0 (now-ms)
3 a (first (filter odd? (map inc (range n))))6 a (first (filter odd? (map inc (range n))))
modified examples/persistent.clj +2 -0
@@ -1,3 +1,5 @@
1+(ns persistent)
2+
13 ;; build a big vector by conj, then read it back
24 (def n 5000)
35 (def v (reduce conj [] (range n)))
@@ -1,3 +1,5 @@
1+(ns persistent)
2+
1 ;; build a big vector by conj, then read it back3 ;; build a big vector by conj, then read it back
2 (def n 5000)4 (def n 5000)
3 (def v (reduce conj [] (range n)))5 (def v (reduce conj [] (range n)))
renamed examples/persistent_bench.clj +3 -0
similarity index 89%
rename from examples/persistent-bench.clj
rename to examples/persistent_bench.clj
@@ -1,3 +1,6 @@
1+(ns persistent-bench
2+ (:require [clonim.core :refer [now-ms]]))
3+
14 (def n 20000)
25 (let [t0 (now-ms)
36 v (reduce conj [] (range n))
similarity index 89%
rename from examples/persistent-bench.clj
rename to examples/persistent_bench.clj
@@ -1,3 +1,6 @@
1+(ns persistent-bench
2+ (:require [clonim.core :refer [now-ms]]))
3+
1 (def n 20000)4 (def n 20000)
2 (let [t0 (now-ms)5 (let [t0 (now-ms)
3 v (reduce conj [] (range n))6 v (reduce conj [] (range n))
modified examples/rebinding.clj +5 -3
@@ -1,13 +1,15 @@
1+(ns rebinding)
2+
13 ;; a fn redefined later: every call site sees the definition in force
24 (defn f [x] (+ x 1))
35 (println (f 10)) ; 11
46 (def f (fn [x] (* x 100)))
57 (println (f 10)) ; 1000
68
7-;; redefining a primitive defeats specialisation for the whole program, so the
8-;; loop below must observe the new + even though it was compiled earlier
9+;; A later user/+ shadows core for subsequent forms, but cannot change the
10+;; clojure.core/+ reference already resolved in sum.
911 (defn sum [n] (loop [i 0 acc 0] (if (< i n) (recur (inc i) (+ acc i)) acc)))
1012 (println (sum 5)) ; 0+1+2+3+4 = 10
1113 (def + (fn [a b] (* a b)))
12-(println (sum 5)) ; + is now *, so acc stays 0
14+(println (sum 5)) ; still uses clojure.core/+: 10
1315 (println (+ 3 4)) ; 12
@@ -1,13 +1,15 @@
1+(ns rebinding)
2+
1 ;; a fn redefined later: every call site sees the definition in force3 ;; a fn redefined later: every call site sees the definition in force
2 (defn f [x] (+ x 1))4 (defn f [x] (+ x 1))
3 (println (f 10)) ; 115 (println (f 10)) ; 11
4 (def f (fn [x] (* x 100)))6 (def f (fn [x] (* x 100)))
5 (println (f 10)) ; 10007 (println (f 10)) ; 1000
6 8
7-;; redefining a primitive defeats specialisation for the whole program, so the9+;; A later user/+ shadows core for subsequent forms, but cannot change the
8-;; loop below must observe the new + even though it was compiled earlier10+;; clojure.core/+ reference already resolved in sum.
9 (defn sum [n] (loop [i 0 acc 0] (if (< i n) (recur (inc i) (+ acc i)) acc)))11 (defn sum [n] (loop [i 0 acc 0] (if (< i n) (recur (inc i) (+ acc i)) acc)))
10 (println (sum 5)) ; 0+1+2+3+4 = 1012 (println (sum 5)) ; 0+1+2+3+4 = 10
11 (def + (fn [a b] (* a b)))13 (def + (fn [a b] (* a b)))
12-(println (sum 5)) ; + is now *, so acc stays 014+(println (sum 5)) ; still uses clojure.core/+: 10
13 (println (+ 3 4)) ; 1215 (println (+ 3 4)) ; 12
modified examples/seqlib.clj +2 -0
@@ -1,3 +1,5 @@
1+(ns seqlib)
2+
13 ;; range: arities, steps, empty and negative
24 (println (range 5) (range 2 5) (range 0 10 3) (range 5 0 -2))
35 (println (range 0) (range 5 5) (range 5 0) (range 0 5 -1))
@@ -1,3 +1,5 @@
1+(ns seqlib)
2+
1 ;; range: arities, steps, empty and negative3 ;; range: arities, steps, empty and negative
2 (println (range 5) (range 2 5) (range 0 10 3) (range 5 0 -2))4 (println (range 5) (range 2 5) (range 0 10 3) (range 5 0 -2))
3 (println (range 0) (range 5 5) (range 5 0) (range 0 5 -1))5 (println (range 0) (range 5 5) (range 5 0) (range 0 5 -1))
modified examples/typespec.clj +1 -1
@@ -24,7 +24,7 @@
2424 ;; comparison chain of non-numbers must not become an int compare
2525 (println (= "a" "a") (= :k :k) (= [1] [1]) (not= 1 2)) ; true true true true
2626 ;; division by zero still reports, rather than trapping
27-(println (try (quot 1 0) (catch Exception e "caught")))
27+(println (try (quot 1 0) (catch Exception _e "caught")))
2828 ;; a fn whose recur makes a parameter non-integral must not specialise
2929 (defn k [x n] (if (< n 3) (recur (str x "a") (+ n 1)) x))
3030 (println (k "" 0)) ; aaa
@@ -24,7 +24,7 @@
24 ;; comparison chain of non-numbers must not become an int compare24 ;; comparison chain of non-numbers must not become an int compare
25 (println (= "a" "a") (= :k :k) (= [1] [1]) (not= 1 2)) ; true true true true25 (println (= "a" "a") (= :k :k) (= [1] [1]) (not= 1 2)) ; true true true true
26 ;; division by zero still reports, rather than trapping26 ;; division by zero still reports, rather than trapping
27-(println (try (quot 1 0) (catch Exception e "caught")))27+(println (try (quot 1 0) (catch Exception _e "caught")))
28 ;; a fn whose recur makes a parameter non-integral must not specialise28 ;; a fn whose recur makes a parameter non-integral must not specialise
29 (defn k [x n] (if (< n 3) (recur (str x "a") (+ n 1)) x))29 (defn k [x n] (if (< n 3) (recur (str x "a") (+ n 1)) x))
30 (println (k "" 0)) ; aaa30 (println (k "" 0)) ; aaa
modified justfile +2 -2
@@ -28,8 +28,8 @@ compile file: build
2828
2929 # Benchmarks: persistent-collection writes, and lazy early exit
3030 bench: release
31- ./{{bin}} run examples/persistent-bench.clj -d
32- ./{{bin}} run examples/lazy-bench.clj -d
31+ ./{{bin}} run examples/persistent_bench.clj -d
32+ ./{{bin}} run examples/lazy_bench.clj -d
3333
3434 # Re-record tests/<name>.expected from current output
3535 accept: build
@@ -28,8 +28,8 @@ compile file: build
28 28
29 # Benchmarks: persistent-collection writes, and lazy early exit29 # Benchmarks: persistent-collection writes, and lazy early exit
30 bench: release30 bench: release
31- ./{{bin}} run examples/persistent-bench.clj -d31+ ./{{bin}} run examples/persistent_bench.clj -d
32- ./{{bin}} run examples/lazy-bench.clj -d32+ ./{{bin}} run examples/lazy_bench.clj -d
33 33
34 # Re-record tests/<name>.expected from current output34 # Re-record tests/<name>.expected from current output
35 accept: build35 accept: build
modified src/clonim.nim +14 -3
@@ -16,6 +16,7 @@ usage:
1616
1717 options:
1818 -o <path> output binary path (build)
19+ --source-path <path> add a library source root (repeatable)
1920 -v show the nim build command and timings
2021 -d build with -d:release (default for build, off for run)"""
2122 quit(1)
@@ -27,10 +28,12 @@ proc srcDir(): string =
2728 if fileExists(cand / "runtime.nim"): return cand
2829 getAppDir()
2930
31+
3032 proc buildKey(nimSrc: string, release: bool): string =
3133 ## Identifies everything the produced binary depends on. Any change here
3234 ## invalidates the cached binary for a source file.
3335 var h: Hash = hash(nimSrc) !& hash(release)
36+
3437 for f in walkFiles(srcDir() / "*.nim"):
3538 h = h !& hash(readFile(f))
3639 let nimExe = findExe("nim")
@@ -50,6 +53,7 @@ proc main() =
5053 var outBin = ""
5154 var verbose = false
5255 var release = cmd == "build"
56+ var sourceRoots: seq[string] = @[]
5357 var i = 2
5458 while i < argv.len:
5559 case argv[i]
@@ -57,6 +61,10 @@ proc main() =
5761 inc i
5862 if i >= argv.len: usage()
5963 outBin = argv[i]
64+ of "--source-path":
65+ inc i
66+ if i >= argv.len: usage()
67+ sourceRoots.add argv[i].absolutePath
6068 of "-v": verbose = true
6169 of "-d": release = true
6270 else: usage()
@@ -65,9 +73,12 @@ proc main() =
6573 let t0 = epochTime()
6674 var nimSrc = ""
6775 try:
68- nimSrc = compileSource(readFile(file))
69- except CljError as e:
70- stderr.writeLine("clonim: " & e.msg)
76+ # Source-level libraries are loaded only by an explicit require form.
77+ sourceRoots.add @[getCurrentDir(), file.absolutePath.parentDir,
78+ srcDir().parentDir / "stdlib"]
79+ nimSrc = compileSource(readFile(file), sourceRoots)
80+ except CljError, IOError, OSError:
81+ stderr.writeLine("clonim: " & getCurrentExceptionMsg())
7182 quit(1)
7283 let tCompile = epochTime() - t0
7384
@@ -16,6 +16,7 @@ usage:
16 16
17 options:17 options:
18 -o <path> output binary path (build)18 -o <path> output binary path (build)
19+ --source-path <path> add a library source root (repeatable)
19 -v show the nim build command and timings20 -v show the nim build command and timings
20 -d build with -d:release (default for build, off for run)"""21 -d build with -d:release (default for build, off for run)"""
21 quit(1)22 quit(1)
@@ -27,10 +28,12 @@ proc srcDir(): string =
27 if fileExists(cand / "runtime.nim"): return cand28 if fileExists(cand / "runtime.nim"): return cand
28 getAppDir()29 getAppDir()
29 30
31+
30 proc buildKey(nimSrc: string, release: bool): string =32 proc buildKey(nimSrc: string, release: bool): string =
31 ## Identifies everything the produced binary depends on. Any change here33 ## Identifies everything the produced binary depends on. Any change here
32 ## invalidates the cached binary for a source file.34 ## invalidates the cached binary for a source file.
33 var h: Hash = hash(nimSrc) !& hash(release)35 var h: Hash = hash(nimSrc) !& hash(release)
36+
34 for f in walkFiles(srcDir() / "*.nim"):37 for f in walkFiles(srcDir() / "*.nim"):
35 h = h !& hash(readFile(f))38 h = h !& hash(readFile(f))
36 let nimExe = findExe("nim")39 let nimExe = findExe("nim")
@@ -50,6 +53,7 @@ proc main() =
50 var outBin = ""53 var outBin = ""
51 var verbose = false54 var verbose = false
52 var release = cmd == "build"55 var release = cmd == "build"
56+ var sourceRoots: seq[string] = @[]
53 var i = 257 var i = 2
54 while i < argv.len:58 while i < argv.len:
55 case argv[i]59 case argv[i]
@@ -57,6 +61,10 @@ proc main() =
57 inc i61 inc i
58 if i >= argv.len: usage()62 if i >= argv.len: usage()
59 outBin = argv[i]63 outBin = argv[i]
64+ of "--source-path":
65+ inc i
66+ if i >= argv.len: usage()
67+ sourceRoots.add argv[i].absolutePath
60 of "-v": verbose = true68 of "-v": verbose = true
61 of "-d": release = true69 of "-d": release = true
62 else: usage()70 else: usage()
@@ -65,9 +73,12 @@ proc main() =
65 let t0 = epochTime()73 let t0 = epochTime()
66 var nimSrc = ""74 var nimSrc = ""
67 try:75 try:
68- nimSrc = compileSource(readFile(file))76+ # Source-level libraries are loaded only by an explicit require form.
69- except CljError as e:77+ sourceRoots.add @[getCurrentDir(), file.absolutePath.parentDir,
70- stderr.writeLine("clonim: " & e.msg)78+ srcDir().parentDir / "stdlib"]
79+ nimSrc = compileSource(readFile(file), sourceRoots)
80+ except CljError, IOError, OSError:
81+ stderr.writeLine("clonim: " & getCurrentExceptionMsg())
71 quit(1)82 quit(1)
72 let tCompile = epochTime() - t083 let tCompile = epochTime() - t0
73 84
modified src/compiler.nim +43 -38
@@ -4,7 +4,7 @@
44 ## special forms (everything else is expanded), then emit host-language code
55 ## and let the host compiler do register allocation, inlining and codegen.
66 import std/[tables, strutils, sets]
7-import runtime, reader
7+import runtime, namespaces
88
99 type
1010 ## A fn whose arity is known at the call site, so it can be reached as a
@@ -75,10 +75,15 @@ proc lookupDirect(env: Env, name: string, arity: int): Direct =
7575 e = e.parent
7676 Direct()
7777
78+proc coreName(name: string): string =
79+ ## Normalize only core names for dispatch, never var or local identity.
80+ ## Other namespaces must not acquire core optimizations by basename.
81+ if name.startsWith("clojure.core/"): name[13 .. ^1] else: name
82+
7883 proc primStable(c: Ctx, name: string): bool =
79- ## A core builtin still holds its registerCore value everywhere: no def in
80- ## this program targets the name at all.
81- c.defCounts[name] == 0
84+ ## The runtime bridge aliases both core spellings to the same cell.
85+ let base = coreName(name)
86+ c.defCounts[base] == 0 and c.defCounts["clojure.core/" & base] == 0
8287
8388 proc fnStable(c: Ctx, name: string): bool =
8489 ## A user fn is reached by exactly one definition, so the one a call site was
@@ -308,10 +313,10 @@ proc collectRecurs(forms: seq[Value], into: var seq[seq[Value]]) =
308313 if f.isNil or f.kind != kList or f.items.len == 0: continue
309314 let head = f.items[0]
310315 if head.kind == kSymbol:
311- if head.s == "recur":
316+ if coreName(head.s) == "recur":
312317 into.add f.items[1 .. ^1]
313318 continue
314- if head.s in ["loop", "loop*", "fn", "fn*", "defn", "defn-"]: continue
319+ if coreName(head.s) in ["loop", "loop*", "fn", "fn*", "defn", "defn-"]: continue
315320 collectRecurs(f.items, into)
316321
317322 proc usesIntPrim(c: Ctx, forms: seq[Value]): bool =
@@ -322,8 +327,8 @@ proc usesIntPrim(c: Ctx, forms: seq[Value]): bool =
322327 if f.isNil or f.kind != kList or f.items.len == 0: continue
323328 let head = f.items[0]
324329 if head.kind == kSymbol and c.primStable(head.s) and
325- (intOps.hasKey(head.s) or intCalls.hasKey(head.s) or
326- cmpOps.hasKey(head.s) or head.s == "inc" or head.s == "dec"):
330+ (intOps.hasKey(coreName(head.s)) or intCalls.hasKey(coreName(head.s)) or
331+ cmpOps.hasKey(coreName(head.s)) or coreName(head.s) in ["inc", "dec"]):
327332 return true
328333 if usesIntPrim(c, f.items): return true
329334 false
@@ -495,7 +500,7 @@ proc genLet(bindings: Value, body: seq[Value], dst: string, env: Env, c: Ctx) =
495500 lenv.locals[restSym] = id
496501 break
497502 let id = c.gensym("l" & mangle(symName(p)))
498- c.line("var " & id & ": Value = call(getVar(\"nth\"), [" & v & ", mkInt(" &
503+ c.line("var " & id & ": Value = call(getVar(\"clojure.core/nth\"), [" & v & ", mkInt(" &
499504 $idx & "), NilV])")
500505 lenv.locals[symName(p)] = id
501506 inc idx; inc j
@@ -506,14 +511,14 @@ proc genLet(bindings: Value, body: seq[Value], dst: string, env: Env, c: Ctx) =
506511 for ks in valForm.items:
507512 let nm = symName(ks)
508513 let id = c.gensym("l" & mangle(nm))
509- c.line("var " & id & ": Value = call(getVar(\"get\"), [" & v &
514+ c.line("var " & id & ": Value = call(getVar(\"clojure.core/get\"), [" & v &
510515 ", mkKeyword(" & nimStr(nm) & ")])")
511516 lenv.locals[nm] = id
512517 else:
513518 let nm = symName(k)
514519 let id = c.gensym("l" & mangle(nm))
515520 let kv = genExpr(valForm, lenv, c)
516- c.line("var " & id & ": Value = call(getVar(\"get\"), [" & v & ", " & kv & "])")
521+ c.line("var " & id & ": Value = call(getVar(\"clojure.core/get\"), [" & v & ", " & kv & "])")
517522 lenv.locals[nm] = id
518523 else:
519524 err("Unsupported binding form: " & prStr(target))
@@ -589,15 +594,15 @@ proc fusableStage(c: Ctx, env: Env, f: Value): bool =
589594 if f.kind != kList or f.items.len != 3: return false
590595 let h = f.items[0]
591596 if h.kind != kSymbol: return false
592- if h.s notin ["map", "filter", "remove"]: return false
597+ if coreName(h.s) notin ["map", "filter", "remove"]: return false
593598 env.lookup(h.s).len == 0 and c.primStable(h.s)
594599
595600 proc fusableCall(c: Ctx, env: Env, head: string, args: seq[Value]): bool =
596601 ## Whether this is a pipeline tryFuse will take.
597602 if env.lookup(head).len > 0 or not c.primStable(head): return false
598603 var collIdx = -1
599- if head == "reduce" and args.len in {2, 3}: collIdx = args.len - 1
600- elif head == "count" and args.len == 1: collIdx = 0
604+ if coreName(head) == "reduce" and args.len in {2, 3}: collIdx = args.len - 1
605+ elif coreName(head) == "count" and args.len == 1: collIdx = 0
601606 else: return false
602607 c.fusableStage(env, args[collIdx])
603608
@@ -609,7 +614,7 @@ proc tryFuse(c: Ctx, env: Env, head: string, args: seq[Value],
609614 ## memoisation -- which makes it safe to run the whole chain as one loop.
610615 ## A named pipeline is left alone: `(def ys (map f xs))` must still cache.
611616 if not c.fusableCall(env, head, args): return false
612- let collIdx = (if head == "count": 0 else: args.len - 1)
617+ let collIdx = (if coreName(head) == "count": 0 else: args.len - 1)
613618
614619 # Operands are emitted in source order: the reducing fn, then each stage's
615620 # fn from outermost in, then the base. That is the order Clojure evaluates
@@ -619,7 +624,7 @@ proc tryFuse(c: Ctx, env: Env, head: string, args: seq[Value],
619624 var ops: seq[string] = @[]
620625 var cur = args[collIdx]
621626 while c.fusableStage(env, cur):
622- let stage = symName(cur.items[0])
627+ let stage = coreName(symName(cur.items[0]))
623628 let fnIdent = genExprTemp(cur.items[1], env, c)
624629 ops.add(if stage == "map": "FusedOp(isMap: true, fn: " & fnIdent & ")"
625630 else: "FusedOp(isMap: false, fn: " & fnIdent & ", keep: " &
@@ -627,7 +632,7 @@ proc tryFuse(c: Ctx, env: Env, head: string, args: seq[Value],
627632 cur = cur.items[2]
628633 let baseIdent = genExprTemp(cur, env, c)
629634 let opsLit = "[" & ops.join(", ") & "]"
630- if head == "count":
635+ if coreName(head) == "count":
631636 c.line(dst & " = fusedCount(" & baseIdent & ", " & opsLit & ")")
632637 elif fixed.len == 1:
633638 c.line(dst & " = fusedReduce(" & fixed[0] & ", NilV, false, " &
@@ -658,7 +663,7 @@ proc genCall(f: Value, args: seq[Value], dst: string, env: Env, c: Ctx) =
658663 (if argIdents.len == 0: "emptyArgs" else: "[" & argIdents.join(", ") & "]") & ")")
659664 c.pop
660665 return
661- let key = f.s & "/" & $args.len
666+ let key = coreName(f.s) & "/" & $args.len
662667 if intrinsics.hasKey(key) and env.lookup(f.s).len == 0 and
663668 c.primStable(f.s):
664669 var argIdents: seq[string] = @[]
@@ -711,7 +716,7 @@ proc intExpr(f: Value, env: Env, c: Ctx): string =
711716 if head.kind != kSymbol: return ""
712717 let args = f.items[1 .. ^1]
713718 # (if c a b) is an int when both arms are
714- if head.s == "if" and args.len == 3:
719+ if coreName(head.s) == "if" and args.len == 3:
715720 let cond = boolExpr(args[0], env, c)
716721 if cond.len == 0: return ""
717722 let a = intExpr(args[1], env, c)
@@ -719,20 +724,20 @@ proc intExpr(f: Value, env: Env, c: Ctx): string =
719724 let b = intExpr(args[2], env, c)
720725 if b.len == 0: return ""
721726 return "(if " & cond & ": " & a & " else: " & b & ")"
722- if specialHeads.contains(head.s): return ""
727+ if specialHeads.contains(coreName(head.s)): return ""
723728 if env.lookup(head.s).len == 0 and c.primStable(head.s):
724- if args.len == 2 and (intOps.hasKey(head.s) or intCalls.hasKey(head.s)):
729+ if args.len == 2 and (intOps.hasKey(coreName(head.s)) or intCalls.hasKey(coreName(head.s))):
725730 let a = intExpr(args[0], env, c)
726731 if a.len == 0: return ""
727732 let b = intExpr(args[1], env, c)
728733 if b.len == 0: return ""
729- if intCalls.hasKey(head.s):
730- return intCalls[head.s] & "(" & a & ", " & b & ")"
731- return "(" & a & " " & intOps[head.s] & " " & b & ")"
732- if args.len == 1 and (head.s == "inc" or head.s == "dec"):
734+ if intCalls.hasKey(coreName(head.s)):
735+ return intCalls[coreName(head.s)] & "(" & a & ", " & b & ")"
736+ return "(" & a & " " & intOps[coreName(head.s)] & " " & b & ")"
737+ if args.len == 1 and (coreName(head.s) == "inc" or coreName(head.s) == "dec"):
733738 let a = intExpr(args[0], env, c)
734739 if a.len == 0: return ""
735- return "(" & a & (if head.s == "inc": " + 1" else: " - 1") & ")"
740+ return "(" & a & (if coreName(head.s) == "inc": " + 1" else: " - 1") & ")"
736741 # a call to an int-specialised fn that returns a raw int64
737742 let (prc, boxes) = env.lookupIntFn(head.s, args.len)
738743 if prc.len > 0 and not boxes:
@@ -750,13 +755,13 @@ proc boolExpr(f: Value, env: Env, c: Ctx): string =
750755 ## A Nim bool expression for a comparison between provable integers.
751756 if f.kind != kList or f.items.len != 3: return ""
752757 let head = f.items[0]
753- if head.kind != kSymbol or not cmpOps.hasKey(head.s): return ""
758+ if head.kind != kSymbol or not cmpOps.hasKey(coreName(head.s)): return ""
754759 if env.lookup(head.s).len > 0 or not c.primStable(head.s): return ""
755760 let a = intExpr(f.items[1], env, c)
756761 if a.len == 0: return ""
757762 let b = intExpr(f.items[2], env, c)
758763 if b.len == 0: return ""
759- "(" & a & " " & cmpOps[head.s] & " " & b & ")"
764+ "(" & a & " " & cmpOps[coreName(head.s)] & " " & b & ")"
760765
761766 proc tryExprs(xs: seq[Value], env: Env, c: Ctx, ids: var seq[string]): bool =
762767 ## All-or-nothing: if any subform needs statements, the caller must fall back
@@ -798,7 +803,7 @@ proc tryExpr(f: Value, env: Env, c: Ctx): string =
798803 if f.items.len == 0: return "mkList(newSeq[Value]())"
799804 let head = f.items[0]
800805 let args = f.items[1 .. ^1]
801- if head.kind == kSymbol and specialHeads.contains(head.s): return ""
806+ if head.kind == kSymbol and specialHeads.contains(coreName(head.s)): return ""
802807 # A fusable pipeline is compiled as statements, so its operands are
803808 # evaluated in source order rather than in Nim argument order.
804809 if head.kind == kSymbol and c.fusableCall(env, head.s, args): return ""
@@ -821,7 +826,7 @@ proc tryExpr(f: Value, env: Env, c: Ctx): string =
821826 # a self-call, or a name only one def form ever targets
822827 return d.prc & "(" & ids.join(", ") & ")"
823828 if d.prc.len == 0:
824- let key = head.s & "/" & $args.len
829+ let key = coreName(head.s) & "/" & $args.len
825830 if intrinsics.hasKey(key) and env.lookup(head.s).len == 0:
826831 if c.primStable(head.s):
827832 return intrinsics[key] & "(" & ids.join(", ") & ")"
@@ -876,7 +881,7 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
876881 let args = f.items[1 .. ^1]
877882 if head.kind == kSymbol and c.tryFuse(env, head.s, args, dst): return
878883 if head.kind == kSymbol:
879- case head.s
884+ case coreName(head.s)
880885 of "quote":
881886 c.line(dst & " = " & quoteLit(args[0]))
882887 return
@@ -1020,12 +1025,12 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
10201025 c.push
10211026 let benv = newEnv(env)
10221027 benv.locals[nm] = id
1023- if head.s == "when-let": genBody(args[1 .. ^1], dst, benv, c)
1028+ if coreName(head.s) == "when-let": genBody(args[1 .. ^1], dst, benv, c)
10241029 else: genInto(args[1], dst, benv, c)
10251030 c.pop
10261031 c.line("else:")
10271032 c.push
1028- if head.s == "if-let" and args.len > 2: genInto(args[2], dst, env, c)
1033+ if coreName(head.s) == "if-let" and args.len > 2: genInto(args[2], dst, env, c)
10291034 else: c.line(dst & " = NilV")
10301035 c.pop
10311036 return
@@ -1129,7 +1134,7 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
11291134 # ------------------------------------------------------------- entry point
11301135 const preamble = """
11311136 ## Generated by clonim. Do not edit.
1132-import runtime, core
1137+import runtime, core, namespaces
11331138
11341139 proc cljMain() =
11351140 """
@@ -1142,7 +1147,7 @@ proc collectDefs(f: Value, into: var CountTable[string]) =
11421147 ## and lets the analyzer trust `+` to be arithmetic.
11431148 if f.isNil or f.kind != kList or f.items.len == 0: return
11441149 let head = f.items[0]
1145- if head.kind == kSymbol and head.s in ["def", "defn", "defn-"] and
1150+ if head.kind == kSymbol and coreName(head.s) in ["def", "defn", "defn-"] and
11461151 f.items.len > 1 and f.items[1].kind == kSymbol:
11471152 into.inc f.items[1].s
11481153 for x in f.items: collectDefs(x, into)
@@ -1160,7 +1165,7 @@ proc compileForms*(forms: seq[Value]): string =
11601165 c.line("var " & t & ": Value = NilV")
11611166 genInto(f, t, env, c)
11621167 c.line("discard " & t)
1163- var src = preamble & " registerCore()\n" & c.prelude.join("\n") & "\n" &
1168+ var src = preamble & " registerCore()\n registerNamespaceCore()\n" & c.prelude.join("\n") & "\n" &
11641169 c.body.join("\n") & "\n\n"
11651170 src &= """
11661171 when isMainModule:
@@ -1172,5 +1177,5 @@ when isMainModule:
11721177 """
11731178 src
11741179
1175-proc compileSource*(src: string): string =
1176- compileForms(readAll(src))
1180+proc compileSource*(src: string, sourceRoots: seq[string] = @[]): string =
1181+ compileForms(resolveSource(src, sourceRoots))
@@ -4,7 +4,7 @@
4 ## special forms (everything else is expanded), then emit host-language code4 ## special forms (everything else is expanded), then emit host-language code
5 ## and let the host compiler do register allocation, inlining and codegen.5 ## and let the host compiler do register allocation, inlining and codegen.
6 import std/[tables, strutils, sets]6 import std/[tables, strutils, sets]
7-import runtime, reader7+import runtime, namespaces
8 8
9 type9 type
10 ## A fn whose arity is known at the call site, so it can be reached as a10 ## A fn whose arity is known at the call site, so it can be reached as a
@@ -75,10 +75,15 @@ proc lookupDirect(env: Env, name: string, arity: int): Direct =
75 e = e.parent75 e = e.parent
76 Direct()76 Direct()
77 77
78+proc coreName(name: string): string =
79+ ## Normalize only core names for dispatch, never var or local identity.
80+ ## Other namespaces must not acquire core optimizations by basename.
81+ if name.startsWith("clojure.core/"): name[13 .. ^1] else: name
82+
78 proc primStable(c: Ctx, name: string): bool =83 proc primStable(c: Ctx, name: string): bool =
79- ## A core builtin still holds its registerCore value everywhere: no def in84+ ## The runtime bridge aliases both core spellings to the same cell.
80- ## this program targets the name at all.85+ let base = coreName(name)
81- c.defCounts[name] == 086+ c.defCounts[base] == 0 and c.defCounts["clojure.core/" & base] == 0
82 87
83 proc fnStable(c: Ctx, name: string): bool =88 proc fnStable(c: Ctx, name: string): bool =
84 ## A user fn is reached by exactly one definition, so the one a call site was89 ## A user fn is reached by exactly one definition, so the one a call site was
@@ -308,10 +313,10 @@ proc collectRecurs(forms: seq[Value], into: var seq[seq[Value]]) =
308 if f.isNil or f.kind != kList or f.items.len == 0: continue313 if f.isNil or f.kind != kList or f.items.len == 0: continue
309 let head = f.items[0]314 let head = f.items[0]
310 if head.kind == kSymbol:315 if head.kind == kSymbol:
311- if head.s == "recur":316+ if coreName(head.s) == "recur":
312 into.add f.items[1 .. ^1]317 into.add f.items[1 .. ^1]
313 continue318 continue
314- if head.s in ["loop", "loop*", "fn", "fn*", "defn", "defn-"]: continue319+ if coreName(head.s) in ["loop", "loop*", "fn", "fn*", "defn", "defn-"]: continue
315 collectRecurs(f.items, into)320 collectRecurs(f.items, into)
316 321
317 proc usesIntPrim(c: Ctx, forms: seq[Value]): bool =322 proc usesIntPrim(c: Ctx, forms: seq[Value]): bool =
@@ -322,8 +327,8 @@ proc usesIntPrim(c: Ctx, forms: seq[Value]): bool =
322 if f.isNil or f.kind != kList or f.items.len == 0: continue327 if f.isNil or f.kind != kList or f.items.len == 0: continue
323 let head = f.items[0]328 let head = f.items[0]
324 if head.kind == kSymbol and c.primStable(head.s) and329 if head.kind == kSymbol and c.primStable(head.s) and
325- (intOps.hasKey(head.s) or intCalls.hasKey(head.s) or330+ (intOps.hasKey(coreName(head.s)) or intCalls.hasKey(coreName(head.s)) or
326- cmpOps.hasKey(head.s) or head.s == "inc" or head.s == "dec"):331+ cmpOps.hasKey(coreName(head.s)) or coreName(head.s) in ["inc", "dec"]):
327 return true332 return true
328 if usesIntPrim(c, f.items): return true333 if usesIntPrim(c, f.items): return true
329 false334 false
@@ -495,7 +500,7 @@ proc genLet(bindings: Value, body: seq[Value], dst: string, env: Env, c: Ctx) =
495 lenv.locals[restSym] = id500 lenv.locals[restSym] = id
496 break501 break
497 let id = c.gensym("l" & mangle(symName(p)))502 let id = c.gensym("l" & mangle(symName(p)))
498- c.line("var " & id & ": Value = call(getVar(\"nth\"), [" & v & ", mkInt(" &503+ c.line("var " & id & ": Value = call(getVar(\"clojure.core/nth\"), [" & v & ", mkInt(" &
499 $idx & "), NilV])")504 $idx & "), NilV])")
500 lenv.locals[symName(p)] = id505 lenv.locals[symName(p)] = id
501 inc idx; inc j506 inc idx; inc j
@@ -506,14 +511,14 @@ proc genLet(bindings: Value, body: seq[Value], dst: string, env: Env, c: Ctx) =
506 for ks in valForm.items:511 for ks in valForm.items:
507 let nm = symName(ks)512 let nm = symName(ks)
508 let id = c.gensym("l" & mangle(nm))513 let id = c.gensym("l" & mangle(nm))
509- c.line("var " & id & ": Value = call(getVar(\"get\"), [" & v &514+ c.line("var " & id & ": Value = call(getVar(\"clojure.core/get\"), [" & v &
510 ", mkKeyword(" & nimStr(nm) & ")])")515 ", mkKeyword(" & nimStr(nm) & ")])")
511 lenv.locals[nm] = id516 lenv.locals[nm] = id
512 else:517 else:
513 let nm = symName(k)518 let nm = symName(k)
514 let id = c.gensym("l" & mangle(nm))519 let id = c.gensym("l" & mangle(nm))
515 let kv = genExpr(valForm, lenv, c)520 let kv = genExpr(valForm, lenv, c)
516- c.line("var " & id & ": Value = call(getVar(\"get\"), [" & v & ", " & kv & "])")521+ c.line("var " & id & ": Value = call(getVar(\"clojure.core/get\"), [" & v & ", " & kv & "])")
517 lenv.locals[nm] = id522 lenv.locals[nm] = id
518 else:523 else:
519 err("Unsupported binding form: " & prStr(target))524 err("Unsupported binding form: " & prStr(target))
@@ -589,15 +594,15 @@ proc fusableStage(c: Ctx, env: Env, f: Value): bool =
589 if f.kind != kList or f.items.len != 3: return false594 if f.kind != kList or f.items.len != 3: return false
590 let h = f.items[0]595 let h = f.items[0]
591 if h.kind != kSymbol: return false596 if h.kind != kSymbol: return false
592- if h.s notin ["map", "filter", "remove"]: return false597+ if coreName(h.s) notin ["map", "filter", "remove"]: return false
593 env.lookup(h.s).len == 0 and c.primStable(h.s)598 env.lookup(h.s).len == 0 and c.primStable(h.s)
594 599
595 proc fusableCall(c: Ctx, env: Env, head: string, args: seq[Value]): bool =600 proc fusableCall(c: Ctx, env: Env, head: string, args: seq[Value]): bool =
596 ## Whether this is a pipeline tryFuse will take.601 ## Whether this is a pipeline tryFuse will take.
597 if env.lookup(head).len > 0 or not c.primStable(head): return false602 if env.lookup(head).len > 0 or not c.primStable(head): return false
598 var collIdx = -1603 var collIdx = -1
599- if head == "reduce" and args.len in {2, 3}: collIdx = args.len - 1604+ if coreName(head) == "reduce" and args.len in {2, 3}: collIdx = args.len - 1
600- elif head == "count" and args.len == 1: collIdx = 0605+ elif coreName(head) == "count" and args.len == 1: collIdx = 0
601 else: return false606 else: return false
602 c.fusableStage(env, args[collIdx])607 c.fusableStage(env, args[collIdx])
603 608
@@ -609,7 +614,7 @@ proc tryFuse(c: Ctx, env: Env, head: string, args: seq[Value],
609 ## memoisation -- which makes it safe to run the whole chain as one loop.614 ## memoisation -- which makes it safe to run the whole chain as one loop.
610 ## A named pipeline is left alone: `(def ys (map f xs))` must still cache.615 ## A named pipeline is left alone: `(def ys (map f xs))` must still cache.
611 if not c.fusableCall(env, head, args): return false616 if not c.fusableCall(env, head, args): return false
612- let collIdx = (if head == "count": 0 else: args.len - 1)617+ let collIdx = (if coreName(head) == "count": 0 else: args.len - 1)
613 618
614 # Operands are emitted in source order: the reducing fn, then each stage's619 # Operands are emitted in source order: the reducing fn, then each stage's
615 # fn from outermost in, then the base. That is the order Clojure evaluates620 # fn from outermost in, then the base. That is the order Clojure evaluates
@@ -619,7 +624,7 @@ proc tryFuse(c: Ctx, env: Env, head: string, args: seq[Value],
619 var ops: seq[string] = @[]624 var ops: seq[string] = @[]
620 var cur = args[collIdx]625 var cur = args[collIdx]
621 while c.fusableStage(env, cur):626 while c.fusableStage(env, cur):
622- let stage = symName(cur.items[0])627+ let stage = coreName(symName(cur.items[0]))
623 let fnIdent = genExprTemp(cur.items[1], env, c)628 let fnIdent = genExprTemp(cur.items[1], env, c)
624 ops.add(if stage == "map": "FusedOp(isMap: true, fn: " & fnIdent & ")"629 ops.add(if stage == "map": "FusedOp(isMap: true, fn: " & fnIdent & ")"
625 else: "FusedOp(isMap: false, fn: " & fnIdent & ", keep: " &630 else: "FusedOp(isMap: false, fn: " & fnIdent & ", keep: " &
@@ -627,7 +632,7 @@ proc tryFuse(c: Ctx, env: Env, head: string, args: seq[Value],
627 cur = cur.items[2]632 cur = cur.items[2]
628 let baseIdent = genExprTemp(cur, env, c)633 let baseIdent = genExprTemp(cur, env, c)
629 let opsLit = "[" & ops.join(", ") & "]"634 let opsLit = "[" & ops.join(", ") & "]"
630- if head == "count":635+ if coreName(head) == "count":
631 c.line(dst & " = fusedCount(" & baseIdent & ", " & opsLit & ")")636 c.line(dst & " = fusedCount(" & baseIdent & ", " & opsLit & ")")
632 elif fixed.len == 1:637 elif fixed.len == 1:
633 c.line(dst & " = fusedReduce(" & fixed[0] & ", NilV, false, " &638 c.line(dst & " = fusedReduce(" & fixed[0] & ", NilV, false, " &
@@ -658,7 +663,7 @@ proc genCall(f: Value, args: seq[Value], dst: string, env: Env, c: Ctx) =
658 (if argIdents.len == 0: "emptyArgs" else: "[" & argIdents.join(", ") & "]") & ")")663 (if argIdents.len == 0: "emptyArgs" else: "[" & argIdents.join(", ") & "]") & ")")
659 c.pop664 c.pop
660 return665 return
661- let key = f.s & "/" & $args.len666+ let key = coreName(f.s) & "/" & $args.len
662 if intrinsics.hasKey(key) and env.lookup(f.s).len == 0 and667 if intrinsics.hasKey(key) and env.lookup(f.s).len == 0 and
663 c.primStable(f.s):668 c.primStable(f.s):
664 var argIdents: seq[string] = @[]669 var argIdents: seq[string] = @[]
@@ -711,7 +716,7 @@ proc intExpr(f: Value, env: Env, c: Ctx): string =
711 if head.kind != kSymbol: return ""716 if head.kind != kSymbol: return ""
712 let args = f.items[1 .. ^1]717 let args = f.items[1 .. ^1]
713 # (if c a b) is an int when both arms are718 # (if c a b) is an int when both arms are
714- if head.s == "if" and args.len == 3:719+ if coreName(head.s) == "if" and args.len == 3:
715 let cond = boolExpr(args[0], env, c)720 let cond = boolExpr(args[0], env, c)
716 if cond.len == 0: return ""721 if cond.len == 0: return ""
717 let a = intExpr(args[1], env, c)722 let a = intExpr(args[1], env, c)
@@ -719,20 +724,20 @@ proc intExpr(f: Value, env: Env, c: Ctx): string =
719 let b = intExpr(args[2], env, c)724 let b = intExpr(args[2], env, c)
720 if b.len == 0: return ""725 if b.len == 0: return ""
721 return "(if " & cond & ": " & a & " else: " & b & ")"726 return "(if " & cond & ": " & a & " else: " & b & ")"
722- if specialHeads.contains(head.s): return ""727+ if specialHeads.contains(coreName(head.s)): return ""
723 if env.lookup(head.s).len == 0 and c.primStable(head.s):728 if env.lookup(head.s).len == 0 and c.primStable(head.s):
724- if args.len == 2 and (intOps.hasKey(head.s) or intCalls.hasKey(head.s)):729+ if args.len == 2 and (intOps.hasKey(coreName(head.s)) or intCalls.hasKey(coreName(head.s))):
725 let a = intExpr(args[0], env, c)730 let a = intExpr(args[0], env, c)
726 if a.len == 0: return ""731 if a.len == 0: return ""
727 let b = intExpr(args[1], env, c)732 let b = intExpr(args[1], env, c)
728 if b.len == 0: return ""733 if b.len == 0: return ""
729- if intCalls.hasKey(head.s):734+ if intCalls.hasKey(coreName(head.s)):
730- return intCalls[head.s] & "(" & a & ", " & b & ")"735+ return intCalls[coreName(head.s)] & "(" & a & ", " & b & ")"
731- return "(" & a & " " & intOps[head.s] & " " & b & ")"736+ return "(" & a & " " & intOps[coreName(head.s)] & " " & b & ")"
732- if args.len == 1 and (head.s == "inc" or head.s == "dec"):737+ if args.len == 1 and (coreName(head.s) == "inc" or coreName(head.s) == "dec"):
733 let a = intExpr(args[0], env, c)738 let a = intExpr(args[0], env, c)
734 if a.len == 0: return ""739 if a.len == 0: return ""
735- return "(" & a & (if head.s == "inc": " + 1" else: " - 1") & ")"740+ return "(" & a & (if coreName(head.s) == "inc": " + 1" else: " - 1") & ")"
736 # a call to an int-specialised fn that returns a raw int64741 # a call to an int-specialised fn that returns a raw int64
737 let (prc, boxes) = env.lookupIntFn(head.s, args.len)742 let (prc, boxes) = env.lookupIntFn(head.s, args.len)
738 if prc.len > 0 and not boxes:743 if prc.len > 0 and not boxes:
@@ -750,13 +755,13 @@ proc boolExpr(f: Value, env: Env, c: Ctx): string =
750 ## A Nim bool expression for a comparison between provable integers.755 ## A Nim bool expression for a comparison between provable integers.
751 if f.kind != kList or f.items.len != 3: return ""756 if f.kind != kList or f.items.len != 3: return ""
752 let head = f.items[0]757 let head = f.items[0]
753- if head.kind != kSymbol or not cmpOps.hasKey(head.s): return ""758+ if head.kind != kSymbol or not cmpOps.hasKey(coreName(head.s)): return ""
754 if env.lookup(head.s).len > 0 or not c.primStable(head.s): return ""759 if env.lookup(head.s).len > 0 or not c.primStable(head.s): return ""
755 let a = intExpr(f.items[1], env, c)760 let a = intExpr(f.items[1], env, c)
756 if a.len == 0: return ""761 if a.len == 0: return ""
757 let b = intExpr(f.items[2], env, c)762 let b = intExpr(f.items[2], env, c)
758 if b.len == 0: return ""763 if b.len == 0: return ""
759- "(" & a & " " & cmpOps[head.s] & " " & b & ")"764+ "(" & a & " " & cmpOps[coreName(head.s)] & " " & b & ")"
760 765
761 proc tryExprs(xs: seq[Value], env: Env, c: Ctx, ids: var seq[string]): bool =766 proc tryExprs(xs: seq[Value], env: Env, c: Ctx, ids: var seq[string]): bool =
762 ## All-or-nothing: if any subform needs statements, the caller must fall back767 ## All-or-nothing: if any subform needs statements, the caller must fall back
@@ -798,7 +803,7 @@ proc tryExpr(f: Value, env: Env, c: Ctx): string =
798 if f.items.len == 0: return "mkList(newSeq[Value]())"803 if f.items.len == 0: return "mkList(newSeq[Value]())"
799 let head = f.items[0]804 let head = f.items[0]
800 let args = f.items[1 .. ^1]805 let args = f.items[1 .. ^1]
801- if head.kind == kSymbol and specialHeads.contains(head.s): return ""806+ if head.kind == kSymbol and specialHeads.contains(coreName(head.s)): return ""
802 # A fusable pipeline is compiled as statements, so its operands are807 # A fusable pipeline is compiled as statements, so its operands are
803 # evaluated in source order rather than in Nim argument order.808 # evaluated in source order rather than in Nim argument order.
804 if head.kind == kSymbol and c.fusableCall(env, head.s, args): return ""809 if head.kind == kSymbol and c.fusableCall(env, head.s, args): return ""
@@ -821,7 +826,7 @@ proc tryExpr(f: Value, env: Env, c: Ctx): string =
821 # a self-call, or a name only one def form ever targets826 # a self-call, or a name only one def form ever targets
822 return d.prc & "(" & ids.join(", ") & ")"827 return d.prc & "(" & ids.join(", ") & ")"
823 if d.prc.len == 0:828 if d.prc.len == 0:
824- let key = head.s & "/" & $args.len829+ let key = coreName(head.s) & "/" & $args.len
825 if intrinsics.hasKey(key) and env.lookup(head.s).len == 0:830 if intrinsics.hasKey(key) and env.lookup(head.s).len == 0:
826 if c.primStable(head.s):831 if c.primStable(head.s):
827 return intrinsics[key] & "(" & ids.join(", ") & ")"832 return intrinsics[key] & "(" & ids.join(", ") & ")"
@@ -876,7 +881,7 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
876 let args = f.items[1 .. ^1]881 let args = f.items[1 .. ^1]
877 if head.kind == kSymbol and c.tryFuse(env, head.s, args, dst): return882 if head.kind == kSymbol and c.tryFuse(env, head.s, args, dst): return
878 if head.kind == kSymbol:883 if head.kind == kSymbol:
879- case head.s884+ case coreName(head.s)
880 of "quote":885 of "quote":
881 c.line(dst & " = " & quoteLit(args[0]))886 c.line(dst & " = " & quoteLit(args[0]))
882 return887 return
@@ -1020,12 +1025,12 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
1020 c.push1025 c.push
1021 let benv = newEnv(env)1026 let benv = newEnv(env)
1022 benv.locals[nm] = id1027 benv.locals[nm] = id
1023- if head.s == "when-let": genBody(args[1 .. ^1], dst, benv, c)1028+ if coreName(head.s) == "when-let": genBody(args[1 .. ^1], dst, benv, c)
1024 else: genInto(args[1], dst, benv, c)1029 else: genInto(args[1], dst, benv, c)
1025 c.pop1030 c.pop
1026 c.line("else:")1031 c.line("else:")
1027 c.push1032 c.push
1028- if head.s == "if-let" and args.len > 2: genInto(args[2], dst, env, c)1033+ if coreName(head.s) == "if-let" and args.len > 2: genInto(args[2], dst, env, c)
1029 else: c.line(dst & " = NilV")1034 else: c.line(dst & " = NilV")
1030 c.pop1035 c.pop
1031 return1036 return
@@ -1129,7 +1134,7 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
1129 # ------------------------------------------------------------- entry point1134 # ------------------------------------------------------------- entry point
1130 const preamble = """1135 const preamble = """
1131 ## Generated by clonim. Do not edit.1136 ## Generated by clonim. Do not edit.
1132-import runtime, core1137+import runtime, core, namespaces
1133 1138
1134 proc cljMain() =1139 proc cljMain() =
1135 """1140 """
@@ -1142,7 +1147,7 @@ proc collectDefs(f: Value, into: var CountTable[string]) =
1142 ## and lets the analyzer trust `+` to be arithmetic.1147 ## and lets the analyzer trust `+` to be arithmetic.
1143 if f.isNil or f.kind != kList or f.items.len == 0: return1148 if f.isNil or f.kind != kList or f.items.len == 0: return
1144 let head = f.items[0]1149 let head = f.items[0]
1145- if head.kind == kSymbol and head.s in ["def", "defn", "defn-"] and1150+ if head.kind == kSymbol and coreName(head.s) in ["def", "defn", "defn-"] and
1146 f.items.len > 1 and f.items[1].kind == kSymbol:1151 f.items.len > 1 and f.items[1].kind == kSymbol:
1147 into.inc f.items[1].s1152 into.inc f.items[1].s
1148 for x in f.items: collectDefs(x, into)1153 for x in f.items: collectDefs(x, into)
@@ -1160,7 +1165,7 @@ proc compileForms*(forms: seq[Value]): string =
1160 c.line("var " & t & ": Value = NilV")1165 c.line("var " & t & ": Value = NilV")
1161 genInto(f, t, env, c)1166 genInto(f, t, env, c)
1162 c.line("discard " & t)1167 c.line("discard " & t)
1163- var src = preamble & " registerCore()\n" & c.prelude.join("\n") & "\n" &1168+ var src = preamble & " registerCore()\n registerNamespaceCore()\n" & c.prelude.join("\n") & "\n" &
1164 c.body.join("\n") & "\n\n"1169 c.body.join("\n") & "\n\n"
1165 src &= """1170 src &= """
1166 when isMainModule:1171 when isMainModule:
@@ -1172,5 +1177,5 @@ when isMainModule:
1172 """1177 """
1173 src1178 src
1174 1179
1175-proc compileSource*(src: string): string =1180+proc compileSource*(src: string, sourceRoots: seq[string] = @[]): string =
1176- compileForms(readAll(src))1181+ compileForms(resolveSource(src, sourceRoots))
modified src/core.nim +2 -1
@@ -540,7 +540,8 @@ proc registerCore*() =
540540 def "slurp", proc (a: openArray[Value]): Value = mkStr(readFile(a[0].s))
541541 def "spit", proc (a: openArray[Value]): Value =
542542 writeFile(a[0].s, str(a[1])); NilV
543- def "now-ms", proc (a: openArray[Value]): Value = mkInt(int64(epochTime() * 1000))
543+ # Host primitive used by the source-level stdlib's now-ms wrapper.
544+ def "*epoch-time-ms*", proc (a: openArray[Value]): Value = mkInt(int64(epochTime() * 1000))
544545
545546 # ---- collections
546547 def "list", proc (a: openArray[Value]): Value = mkList(a)
@@ -540,7 +540,8 @@ proc registerCore*() =
540 def "slurp", proc (a: openArray[Value]): Value = mkStr(readFile(a[0].s))540 def "slurp", proc (a: openArray[Value]): Value = mkStr(readFile(a[0].s))
541 def "spit", proc (a: openArray[Value]): Value =541 def "spit", proc (a: openArray[Value]): Value =
542 writeFile(a[0].s, str(a[1])); NilV542 writeFile(a[0].s, str(a[1])); NilV
543- def "now-ms", proc (a: openArray[Value]): Value = mkInt(int64(epochTime() * 1000))543+ # Host primitive used by the source-level stdlib's now-ms wrapper.
544+ def "*epoch-time-ms*", proc (a: openArray[Value]): Value = mkInt(int64(epochTime() * 1000))
544 545
545 # ---- collections546 # ---- collections
546 def "list", proc (a: openArray[Value]): Value = mkList(a)547 def "list", proc (a: openArray[Value]): Value = mkList(a)
added src/namespaces.nim +415 -0
new file mode 100644
@@ -0,0 +1,415 @@
1+## Static namespaces and dependency loading, before compiler.compileForms.
2+## This pass understands the compiler's fixed macro/binding forms, not arbitrary
3+## user macro expansion. Reader limitations (including syntax-quote and reader
4+## conditionals) still apply. Core discovery temporarily swaps runtime globals;
5+## like that runtime, this module is intended for single-threaded use.
6+## Core var references are canonical clojure.core/name symbols. Generated code
7+## must call registerNamespaceCore() after registerCore() (or register the same
8+## canonical cells itself). Syntax heads remain unqualified for the compiler.
9+import std/[os, tables, sets, strutils]
10+import runtime, reader, core
11+
12+type
13+ Namespace = ref object
14+ name: string
15+ aliases, refers: Table[string, string]
16+ defs, privateDefs, required: HashSet[string]
17+ Resolver = ref object
18+ roots: seq[string]
19+ cores: HashSet[string]
20+ spaces: Table[string, Namespace]
21+ active: seq[string]
22+ loaded: HashSet[string]
23+ output: seq[Value]
24+
25+const syntaxHeads = ["quote", "if", "do", "let", "let*", "loop", "loop*",
26+ "recur", "fn", "fn*", "def", "defn", "defn-", "defmacro", "and", "or",
27+ "when", "when-not", "if-not", "cond", "when-let", "if-let", "->", "->>",
28+ "doseq", "dotimes", "try", "catch", "finally", "comment", "set!", "declare"]
29+
30+# Unlike macros, these heads cannot be shadowed in operator position.
31+const specialForms = ["quote", "if", "do", "let*", "loop*", "recur", "fn*",
32+ "def", "try", "catch", "finally", "set!"]
33+const namespaceHeads = ["ns", "require", "in-ns", "use", "import", "refer",
34+ "load", "load-file"]
35+
36+proc fail(msg: string) {.noreturn.} = err("Namespace error: " & msg)
37+proc isSym(v: Value, s: string): bool = v.kind == kSymbol and v.s == s
38+proc headName(v: Value): string =
39+ if v.kind == kList and v.items.len > 0 and v.items[0].kind == kSymbol:
40+ result = v.items[0].s
41+ if result.startsWith("clojure.core/"): result = result[13 .. ^1]
42+
43+proc simple(v: Value, what: string): string =
44+ if v.kind != kSymbol or v.s.len == 0 or ('/' in v.s and v.s != "/"):
45+ fail("expected unqualified " & what & ", got " & prStr(v))
46+ v.s
47+
48+proc validNamespace(name: string) =
49+ for part in name.split('.'):
50+ if part.len == 0: fail("invalid namespace: " & name)
51+ for ch in part:
52+ if ch notin {'a'..'z', 'A'..'Z', '0'..'9', '_', '-'}:
53+ fail("invalid namespace: " & name)
54+
55+proc registeredCoreNames(): HashSet[string] =
56+ # Isolate registration: do not overwrite caller values or mutate their cells.
57+ let saved = globals
58+ globals = initTable[string, VarCell]()
59+ try:
60+ registerCore()
61+ for name, cell in globals:
62+ if cell.bound and '/' notin name: result.incl name
63+ if globals.hasKey("/"): result.incl "/"
64+ finally:
65+ globals = saved
66+
67+proc registerNamespaceCore*() =
68+ ## Call after registerCore in the generated program, before resolving cells.
69+ ## Both spellings share a cell, so rebinding remains coherent.
70+ let names = registeredCoreNames()
71+ for name in names:
72+ if not hasVar(name): fail("registerCore must precede registerNamespaceCore")
73+ globals["clojure.core/" & name] = varCell(name)
74+
75+proc space(r: Resolver, name: string): Namespace =
76+ if not r.spaces.hasKey(name):
77+ r.spaces[name] = Namespace(name: name)
78+ r.spaces[name]
79+
80+proc checkVar(r: Resolver, ns: Namespace, target, name: string,
81+ allowSyntax = false) =
82+ let canonical = target & "/" & name
83+ if target == "clojure.core":
84+ if name in r.cores or (allowSyntax and name in syntaxHeads): return
85+ fail("no definition " & canonical)
86+ if target != ns.name and target notin ns.required:
87+ fail("namespace " & target & " is not required by " & ns.name)
88+ if not r.spaces.hasKey(target) or name notin r.spaces[target].defs:
89+ fail("no definition " & canonical)
90+ if target != ns.name and name in r.spaces[target].privateDefs:
91+ fail("private var " & canonical & " is inaccessible from " & ns.name)
92+
93+proc qualify(r: Resolver, ns: Namespace, name: string,
94+ locals: HashSet[string]): string =
95+ if name in locals: return name
96+ let slash = name.find('/')
97+ if slash > 0:
98+ let prefix = name[0 ..< slash]
99+ let target = ns.aliases.getOrDefault(prefix, prefix)
100+ let member = name[slash + 1 .. ^1]
101+ r.checkVar(ns, target, member)
102+ return target & "/" & member
103+ if name in ns.defs: return ns.name & "/" & name
104+ if ns.refers.hasKey(name):
105+ let target = ns.refers[name]
106+ let split = target.find('/')
107+ r.checkVar(ns, target[0 ..< split], target[split + 1 .. ^1])
108+ return target
109+ if name in r.cores: return "clojure.core/" & name
110+ fail("unable to resolve symbol " & name & " in " & ns.name)
111+
112+proc syntaxHead(ns: Namespace, v: Value, locals: HashSet[string]): string =
113+ if v.kind != kList or v.items.len == 0 or v.items[0].kind != kSymbol: return
114+ let raw = v.items[0].s
115+ if raw in specialForms: return raw
116+ if raw in locals: return
117+ let slash = raw.find('/')
118+ var canonical: string
119+ if slash > 0:
120+ let prefix = raw[0 ..< slash]
121+ canonical = ns.aliases.getOrDefault(prefix, prefix) & raw[slash .. ^1]
122+ else:
123+ if raw in ns.defs: return
124+ canonical = ns.refers.getOrDefault(raw, "clojure.core/" & raw)
125+ if canonical.startsWith("clojure.core/"):
126+ let name = canonical[13 .. ^1]
127+ if name in syntaxHeads or name in namespaceHeads: return name
128+
129+proc define(ns: Namespace, v: Value, private = false): Value =
130+ let name = simple(v, "definition name")
131+ if ns.refers.hasKey(name) and not ns.refers[name].startsWith("clojure.core/"):
132+ fail("definition conflicts with referred name: " & name)
133+ ns.defs.incl name
134+ if private: ns.privateDefs.incl name
135+ result = mkSymbol(ns.name & "/" & name)
136+
137+proc walk(r: Resolver, ns: Namespace, v: Value, locals: HashSet[string]): Value
138+
139+proc bindNames(v: Value, locals: var HashSet[string]) =
140+ case v.kind
141+ of kSymbol:
142+ if v.s != "&": locals.incl simple(v, "local binding")
143+ of kVector:
144+ for x in v.items:
145+ if x.kind != kKeyword: bindNames(x, locals)
146+ of kMap:
147+ for (k, value) in v.pairs:
148+ if k.kind == kKeyword:
149+ let key = k.s.split('/')[^1]
150+ case key
151+ of "keys", "syms", "strs":
152+ if value.kind != kVector: fail("destructuring requires a vector")
153+ for x in value.items:
154+ if x.kind notin {kSymbol, kStr}: fail("invalid destructuring name")
155+ locals.incl x.s.split('/')[^1]
156+ of "as": bindNames(value, locals)
157+ of "or": discard
158+ else: fail("unsupported destructuring directive: " & k.s)
159+ else: bindNames(k, locals)
160+ else: fail("invalid binding pattern: " & prStr(v))
161+
162+proc pattern(r: Resolver, ns: Namespace, v: Value,
163+ locals: HashSet[string]): Value =
164+ # Patterns are data except for :or defaults, which are expressions.
165+ if v.kind == kMap:
166+ var ps: seq[(Value, Value)]
167+ for (k, value) in v.pairs:
168+ if k.kind == kKeyword and k.s == "or":
169+ if value.kind != kMap: fail(":or requires a map")
170+ var defaults: seq[(Value, Value)]
171+ for (key, expr) in value.pairs:
172+ defaults.add (key, r.walk(ns, expr, locals))
173+ ps.add (k, mkMap(defaults))
174+ elif k.kind != kKeyword:
175+ ps.add (r.pattern(ns, k, locals), value)
176+ else: ps.add (k, value)
177+ return mkMap(ps)
178+ if v.kind == kVector:
179+ var xs: seq[Value]
180+ for x in v.items: xs.add r.pattern(ns, x, locals)
181+ return mkVector(xs)
182+ v
183+
184+proc fnTail(r: Resolver, ns: Namespace, xs: seq[Value],
185+ locals: HashSet[string]): seq[Value] =
186+ if xs.len == 0: fail("function requires parameters")
187+ if xs[0].kind == kVector:
188+ var scope = locals
189+ bindNames(xs[0], scope)
190+ result.add r.pattern(ns, xs[0], locals)
191+ for i in 1 ..< xs.len: result.add r.walk(ns, xs[i], scope)
192+ else:
193+ for clause in xs:
194+ if clause.kind != kList: fail("invalid function arity")
195+ result.add mkList(r.fnTail(ns, clause.items, locals))
196+
197+proc walk(r: Resolver, ns: Namespace, v: Value, locals: HashSet[string]): Value =
198+ case v.kind
199+ of kSymbol:
200+ if ns.syntaxHead(mkList(@[v]), locals) in namespaceHeads:
201+ fail("dynamic namespace operation is unsupported: " & v.s)
202+ return mkSymbol(r.qualify(ns, v.s, locals))
203+ of kMap:
204+ var ps: seq[(Value, Value)]
205+ for (k, value) in v.pairs:
206+ ps.add (r.walk(ns, k, locals), r.walk(ns, value, locals))
207+ return mkMap(ps)
208+ of kVector, kSet:
209+ var xs: seq[Value]
210+ for x in v.items: xs.add r.walk(ns, x, locals)
211+ if v.kind == kVector: return mkVector(xs)
212+ return mkSet(xs)
213+ of kList: discard
214+ else: return v
215+ var xs = v.items
216+ if xs.len == 0: return v
217+ let h = ns.syntaxHead(v, locals)
218+ if h in ["quote", "comment"]:
219+ xs[0] = mkSymbol(h)
220+ return mkList(xs)
221+ if h in namespaceHeads:
222+ fail(h & " is only supported as a static top-level ns/require declaration")
223+ if h in syntaxHeads: xs[0] = mkSymbol(h)
224+ case h
225+ of "fn", "fn*":
226+ var scope = locals
227+ var start = 1
228+ if xs.len > 1 and xs[1].kind == kSymbol:
229+ scope.incl simple(xs[1], "function name")
230+ start = 2
231+ return mkList(xs[0 ..< start] & r.fnTail(ns, xs[start .. ^1], scope))
232+ of "defn", "defn-", "defmacro":
233+ if xs.len < 3: fail(h & " requires a name and parameters")
234+ xs[1] = ns.define(xs[1], h == "defn-")
235+ var start = 2
236+ if start < xs.len and xs[start].kind == kStr: inc start
237+ if start < xs.len and xs[start].kind == kMap: inc start
238+ var scope = locals
239+ if h == "defmacro":
240+ scope.incl "&form"
241+ scope.incl "&env"
242+ return mkList(xs[0 ..< start] & r.fnTail(ns, xs[start .. ^1], scope))
243+ of "def", "declare":
244+ if xs.len < 2: fail(h & " requires a name")
245+ let stop = if h == "declare": xs.len else: 2
246+ for i in 1 ..< stop:
247+ xs[i] = ns.define(xs[i])
248+ for i in stop ..< xs.len: xs[i] = r.walk(ns, xs[i], locals)
249+ return mkList(xs)
250+ of "let", "let*", "loop", "loop*", "when-let", "if-let", "doseq", "dotimes":
251+ if xs.len < 2 or xs[1].kind != kVector: fail(h & " requires bindings")
252+ var bs = xs[1].items
253+ if bs.len mod 2 != 0: fail(h & " requires paired bindings")
254+ if h in ["when-let", "if-let", "doseq", "dotimes"] and bs.len != 2:
255+ fail(h & " supports exactly one binding")
256+ var scope = locals
257+ var i = 0
258+ while i < bs.len:
259+ bs[i + 1] = r.walk(ns, bs[i + 1], scope)
260+ let original = bs[i]
261+ bs[i] = r.pattern(ns, original, scope)
262+ bindNames(original, scope)
263+ i += 2
264+ xs[1] = mkVector(bs)
265+ for j in 2 ..< xs.len:
266+ xs[j] = r.walk(ns, xs[j], if h == "if-let" and j >= 3: locals else: scope)
267+ return mkList(xs)
268+ of "catch":
269+ if xs.len < 3: fail("catch requires a class and binding")
270+ var scope = locals
271+ scope.incl simple(xs[2], "catch binding")
272+ for i in 3 ..< xs.len: xs[i] = r.walk(ns, xs[i], scope)
273+ return mkList(xs)
274+ of "->", "->>":
275+ # Expand before resolving: a bare step can be a syntax head (e.g. do).
276+ if xs.len < 2: fail(h & " requires an expression")
277+ var acc = xs[1]
278+ for i in 2 ..< xs.len:
279+ let step = xs[i]
280+ if step.kind == kList and step.items.len > 0:
281+ let parts = step.items
282+ if h == "->": acc = mkList(@[parts[0], acc] & parts[1 .. ^1])
283+ else: acc = mkList(parts & @[acc])
284+ else: acc = mkList(@[step, acc])
285+ return r.walk(ns, acc, locals)
286+ else: discard
287+ for i in 0 ..< xs.len:
288+ if i == 0 and h in syntaxHeads: continue
289+ if h == "cond" and i mod 2 == 1 and isSym(xs[i], "else"): continue
290+ xs[i] = r.walk(ns, xs[i], locals)
291+ if h.len == 0 and xs[0].kind == kSymbol and xs[0].s in locals and
292+ (xs[0].s in syntaxHeads or xs[0].s in namespaceHeads):
293+ # The compiler dispatches by head spelling before consulting its locals.
294+ # An expression head keeps this a value call without renaming bindings.
295+ xs[0] = mkList(@[mkSymbol("do"), xs[0]])
296+ mkList(xs)
297+
298+proc process(r: Resolver, forms: seq[Value], expected = "")
299+proc load(r: Resolver, name: string) =
300+ validNamespace(name)
301+ if name == "clojure.core": return
302+ if name in r.active: fail("dependency cycle: " & (r.active & @[name]).join(" -> "))
303+ if name in r.loaded: return
304+ let relative = name.replace('.', '/').replace('-', '_')
305+ var path = ""
306+ for root in r.roots:
307+ for ext in [".clj", ".cljc"]:
308+ let candidate = root / (relative & ext)
309+ if fileExists(candidate):
310+ path = candidate
311+ break
312+ if path.len > 0: break
313+ if path.len == 0: fail("cannot find " & name & " in source roots: " & r.roots.join(", "))
314+ try:
315+ r.process(readAll(readFile(path)), name)
316+ except IOError as e:
317+ fail("cannot read " & path & ": " & e.msg)
318+
319+proc requireSpec(r: Resolver, ns: Namespace, spec: Value) =
320+ var xs: seq[Value]
321+ if spec.kind == kSymbol: xs = @[spec]
322+ elif spec.kind == kVector: xs = spec.items
323+ else: fail("require expects a symbol or vector libspec")
324+ if xs.len == 0: fail("empty require libspec")
325+ let name = simple(xs[0], "required namespace")
326+ validNamespace(name)
327+ var alias = ""
328+ var refers: seq[string]
329+ var seen: HashSet[string]
330+ var i = 1
331+ while i < xs.len:
332+ if i + 1 >= xs.len or xs[i].kind != kKeyword: fail("invalid require options")
333+ let option = xs[i].s
334+ if option in seen: fail("duplicate require option: " & option)
335+ seen.incl option
336+ case option
337+ of "as": alias = simple(xs[i + 1], "alias")
338+ of "refer":
339+ if xs[i + 1].kind != kVector: fail(":refer supports only an explicit vector")
340+ for x in xs[i + 1].items: refers.add simple(x, "referred name")
341+ else: fail("unsupported require option: :" & option)
342+ i += 2
343+ r.load(name)
344+ ns.required.incl name
345+ if alias.len > 0:
346+ if ns.aliases.hasKey(alias) and ns.aliases[alias] != name:
347+ fail("conflicting alias: " & alias)
348+ ns.aliases[alias] = name
349+ for local in refers:
350+ r.checkVar(ns, name, local, allowSyntax = true)
351+ let target = name & "/" & local
352+ if local in ns.defs or (ns.refers.hasKey(local) and ns.refers[local] != target):
353+ fail("conflicting referred name: " & local)
354+ ns.refers[local] = target
355+
356+proc flatten(forms: seq[Value]): seq[Value] =
357+ for f in forms:
358+ if headName(f) == "do": result.add flatten(f.items[1 .. ^1])
359+ else: result.add f
360+
361+proc process(r: Resolver, forms: seq[Value], expected = "") =
362+ let flat = flatten(forms)
363+ var name = "user"
364+ var declaration = -1
365+ for i, f in flat:
366+ if headName(f) == "ns":
367+ if declaration >= 0 or i != 0: fail("ns must be the first form and occur once per source")
368+ if f.items.len < 2: fail("ns requires a name")
369+ name = simple(f.items[1], "namespace name")
370+ validNamespace(name)
371+ declaration = i
372+ if expected.len > 0 and (declaration < 0 or name != expected):
373+ fail("source for " & expected & " must declare (ns " & expected & ")")
374+ if name in r.active: fail("dependency cycle: " & (r.active & @[name]).join(" -> "))
375+ let ns = r.space(name)
376+
377+ r.active.add name
378+ try:
379+ for f in flat:
380+ case ns.syntaxHead(f, initHashSet[string]())
381+ of "ns":
382+ let xs = f.items
383+ var start = 2
384+ if start < xs.len and xs[start].kind == kStr: inc start
385+ if start < xs.len and xs[start].kind == kMap: inc start
386+ for i in start ..< xs.len:
387+ let clause = xs[i]
388+ if clause.kind != kList or clause.items.len == 0 or
389+ clause.items[0].kind != kKeyword or clause.items[0].s != "require":
390+ fail("only :require clauses are supported in ns")
391+ for spec in clause.items[1 .. ^1]: r.requireSpec(ns, spec)
392+ of "require":
393+ if f.items.len < 2: fail("require expects quoted libspecs")
394+ for arg in f.items[1 .. ^1]:
395+ if headName(arg) != "quote" or arg.items.len != 2:
396+ fail("dynamic require is unsupported; quote each libspec")
397+ r.requireSpec(ns, arg.items[1])
398+ else:
399+ r.output.add r.walk(ns, f, initHashSet[string]())
400+ r.loaded.incl name
401+ finally:
402+ discard r.active.pop()
403+
404+proc resolveSource*(src: string, sourceRoots: seq[string]): seq[Value] =
405+ ## Read source text and splice dependencies at their first require, once per
406+ ## call. Roots are searched in order, .clj before .cljc. Missing/mismatched
407+ ## namespaces, cycles, unsupported imports and dynamic requires raise CljError.
408+ ## No namespace state or load cache survives a call. Definitions are visible
409+ ## from their declaration onward (including their own initializer/body).
410+ ## Forward references require declare. Qualified references require a direct
411+ ## require, except for the current namespace and implicit clojure.core.
412+ ## Unknown and externally private vars are rejected during analysis.
413+ let r = Resolver(roots: sourceRoots, cores: registeredCoreNames())
414+ r.process(readAll(src))
415+ r.output
new file mode 100644
@@ -0,0 +1,415 @@
1+## Static namespaces and dependency loading, before compiler.compileForms.
2+## This pass understands the compiler's fixed macro/binding forms, not arbitrary
3+## user macro expansion. Reader limitations (including syntax-quote and reader
4+## conditionals) still apply. Core discovery temporarily swaps runtime globals;
5+## like that runtime, this module is intended for single-threaded use.
6+## Core var references are canonical clojure.core/name symbols. Generated code
7+## must call registerNamespaceCore() after registerCore() (or register the same
8+## canonical cells itself). Syntax heads remain unqualified for the compiler.
9+import std/[os, tables, sets, strutils]
10+import runtime, reader, core
11+
12+type
13+ Namespace = ref object
14+ name: string
15+ aliases, refers: Table[string, string]
16+ defs, privateDefs, required: HashSet[string]
17+ Resolver = ref object
18+ roots: seq[string]
19+ cores: HashSet[string]
20+ spaces: Table[string, Namespace]
21+ active: seq[string]
22+ loaded: HashSet[string]
23+ output: seq[Value]
24+
25+const syntaxHeads = ["quote", "if", "do", "let", "let*", "loop", "loop*",
26+ "recur", "fn", "fn*", "def", "defn", "defn-", "defmacro", "and", "or",
27+ "when", "when-not", "if-not", "cond", "when-let", "if-let", "->", "->>",
28+ "doseq", "dotimes", "try", "catch", "finally", "comment", "set!", "declare"]
29+
30+# Unlike macros, these heads cannot be shadowed in operator position.
31+const specialForms = ["quote", "if", "do", "let*", "loop*", "recur", "fn*",
32+ "def", "try", "catch", "finally", "set!"]
33+const namespaceHeads = ["ns", "require", "in-ns", "use", "import", "refer",
34+ "load", "load-file"]
35+
36+proc fail(msg: string) {.noreturn.} = err("Namespace error: " & msg)
37+proc isSym(v: Value, s: string): bool = v.kind == kSymbol and v.s == s
38+proc headName(v: Value): string =
39+ if v.kind == kList and v.items.len > 0 and v.items[0].kind == kSymbol:
40+ result = v.items[0].s
41+ if result.startsWith("clojure.core/"): result = result[13 .. ^1]
42+
43+proc simple(v: Value, what: string): string =
44+ if v.kind != kSymbol or v.s.len == 0 or ('/' in v.s and v.s != "/"):
45+ fail("expected unqualified " & what & ", got " & prStr(v))
46+ v.s
47+
48+proc validNamespace(name: string) =
49+ for part in name.split('.'):
50+ if part.len == 0: fail("invalid namespace: " & name)
51+ for ch in part:
52+ if ch notin {'a'..'z', 'A'..'Z', '0'..'9', '_', '-'}:
53+ fail("invalid namespace: " & name)
54+
55+proc registeredCoreNames(): HashSet[string] =
56+ # Isolate registration: do not overwrite caller values or mutate their cells.
57+ let saved = globals
58+ globals = initTable[string, VarCell]()
59+ try:
60+ registerCore()
61+ for name, cell in globals:
62+ if cell.bound and '/' notin name: result.incl name
63+ if globals.hasKey("/"): result.incl "/"
64+ finally:
65+ globals = saved
66+
67+proc registerNamespaceCore*() =
68+ ## Call after registerCore in the generated program, before resolving cells.
69+ ## Both spellings share a cell, so rebinding remains coherent.
70+ let names = registeredCoreNames()
71+ for name in names:
72+ if not hasVar(name): fail("registerCore must precede registerNamespaceCore")
73+ globals["clojure.core/" & name] = varCell(name)
74+
75+proc space(r: Resolver, name: string): Namespace =
76+ if not r.spaces.hasKey(name):
77+ r.spaces[name] = Namespace(name: name)
78+ r.spaces[name]
79+
80+proc checkVar(r: Resolver, ns: Namespace, target, name: string,
81+ allowSyntax = false) =
82+ let canonical = target & "/" & name
83+ if target == "clojure.core":
84+ if name in r.cores or (allowSyntax and name in syntaxHeads): return
85+ fail("no definition " & canonical)
86+ if target != ns.name and target notin ns.required:
87+ fail("namespace " & target & " is not required by " & ns.name)
88+ if not r.spaces.hasKey(target) or name notin r.spaces[target].defs:
89+ fail("no definition " & canonical)
90+ if target != ns.name and name in r.spaces[target].privateDefs:
91+ fail("private var " & canonical & " is inaccessible from " & ns.name)
92+
93+proc qualify(r: Resolver, ns: Namespace, name: string,
94+ locals: HashSet[string]): string =
95+ if name in locals: return name
96+ let slash = name.find('/')
97+ if slash > 0:
98+ let prefix = name[0 ..< slash]
99+ let target = ns.aliases.getOrDefault(prefix, prefix)
100+ let member = name[slash + 1 .. ^1]
101+ r.checkVar(ns, target, member)
102+ return target & "/" & member
103+ if name in ns.defs: return ns.name & "/" & name
104+ if ns.refers.hasKey(name):
105+ let target = ns.refers[name]
106+ let split = target.find('/')
107+ r.checkVar(ns, target[0 ..< split], target[split + 1 .. ^1])
108+ return target
109+ if name in r.cores: return "clojure.core/" & name
110+ fail("unable to resolve symbol " & name & " in " & ns.name)
111+
112+proc syntaxHead(ns: Namespace, v: Value, locals: HashSet[string]): string =
113+ if v.kind != kList or v.items.len == 0 or v.items[0].kind != kSymbol: return
114+ let raw = v.items[0].s
115+ if raw in specialForms: return raw
116+ if raw in locals: return
117+ let slash = raw.find('/')
118+ var canonical: string
119+ if slash > 0:
120+ let prefix = raw[0 ..< slash]
121+ canonical = ns.aliases.getOrDefault(prefix, prefix) & raw[slash .. ^1]
122+ else:
123+ if raw in ns.defs: return
124+ canonical = ns.refers.getOrDefault(raw, "clojure.core/" & raw)
125+ if canonical.startsWith("clojure.core/"):
126+ let name = canonical[13 .. ^1]
127+ if name in syntaxHeads or name in namespaceHeads: return name
128+
129+proc define(ns: Namespace, v: Value, private = false): Value =
130+ let name = simple(v, "definition name")
131+ if ns.refers.hasKey(name) and not ns.refers[name].startsWith("clojure.core/"):
132+ fail("definition conflicts with referred name: " & name)
133+ ns.defs.incl name
134+ if private: ns.privateDefs.incl name
135+ result = mkSymbol(ns.name & "/" & name)
136+
137+proc walk(r: Resolver, ns: Namespace, v: Value, locals: HashSet[string]): Value
138+
139+proc bindNames(v: Value, locals: var HashSet[string]) =
140+ case v.kind
141+ of kSymbol:
142+ if v.s != "&": locals.incl simple(v, "local binding")
143+ of kVector:
144+ for x in v.items:
145+ if x.kind != kKeyword: bindNames(x, locals)
146+ of kMap:
147+ for (k, value) in v.pairs:
148+ if k.kind == kKeyword:
149+ let key = k.s.split('/')[^1]
150+ case key
151+ of "keys", "syms", "strs":
152+ if value.kind != kVector: fail("destructuring requires a vector")
153+ for x in value.items:
154+ if x.kind notin {kSymbol, kStr}: fail("invalid destructuring name")
155+ locals.incl x.s.split('/')[^1]
156+ of "as": bindNames(value, locals)
157+ of "or": discard
158+ else: fail("unsupported destructuring directive: " & k.s)
159+ else: bindNames(k, locals)
160+ else: fail("invalid binding pattern: " & prStr(v))
161+
162+proc pattern(r: Resolver, ns: Namespace, v: Value,
163+ locals: HashSet[string]): Value =
164+ # Patterns are data except for :or defaults, which are expressions.
165+ if v.kind == kMap:
166+ var ps: seq[(Value, Value)]
167+ for (k, value) in v.pairs:
168+ if k.kind == kKeyword and k.s == "or":
169+ if value.kind != kMap: fail(":or requires a map")
170+ var defaults: seq[(Value, Value)]
171+ for (key, expr) in value.pairs:
172+ defaults.add (key, r.walk(ns, expr, locals))
173+ ps.add (k, mkMap(defaults))
174+ elif k.kind != kKeyword:
175+ ps.add (r.pattern(ns, k, locals), value)
176+ else: ps.add (k, value)
177+ return mkMap(ps)
178+ if v.kind == kVector:
179+ var xs: seq[Value]
180+ for x in v.items: xs.add r.pattern(ns, x, locals)
181+ return mkVector(xs)
182+ v
183+
184+proc fnTail(r: Resolver, ns: Namespace, xs: seq[Value],
185+ locals: HashSet[string]): seq[Value] =
186+ if xs.len == 0: fail("function requires parameters")
187+ if xs[0].kind == kVector:
188+ var scope = locals
189+ bindNames(xs[0], scope)
190+ result.add r.pattern(ns, xs[0], locals)
191+ for i in 1 ..< xs.len: result.add r.walk(ns, xs[i], scope)
192+ else:
193+ for clause in xs:
194+ if clause.kind != kList: fail("invalid function arity")
195+ result.add mkList(r.fnTail(ns, clause.items, locals))
196+
197+proc walk(r: Resolver, ns: Namespace, v: Value, locals: HashSet[string]): Value =
198+ case v.kind
199+ of kSymbol:
200+ if ns.syntaxHead(mkList(@[v]), locals) in namespaceHeads:
201+ fail("dynamic namespace operation is unsupported: " & v.s)
202+ return mkSymbol(r.qualify(ns, v.s, locals))
203+ of kMap:
204+ var ps: seq[(Value, Value)]
205+ for (k, value) in v.pairs:
206+ ps.add (r.walk(ns, k, locals), r.walk(ns, value, locals))
207+ return mkMap(ps)
208+ of kVector, kSet:
209+ var xs: seq[Value]
210+ for x in v.items: xs.add r.walk(ns, x, locals)
211+ if v.kind == kVector: return mkVector(xs)
212+ return mkSet(xs)
213+ of kList: discard
214+ else: return v
215+ var xs = v.items
216+ if xs.len == 0: return v
217+ let h = ns.syntaxHead(v, locals)
218+ if h in ["quote", "comment"]:
219+ xs[0] = mkSymbol(h)
220+ return mkList(xs)
221+ if h in namespaceHeads:
222+ fail(h & " is only supported as a static top-level ns/require declaration")
223+ if h in syntaxHeads: xs[0] = mkSymbol(h)
224+ case h
225+ of "fn", "fn*":
226+ var scope = locals
227+ var start = 1
228+ if xs.len > 1 and xs[1].kind == kSymbol:
229+ scope.incl simple(xs[1], "function name")
230+ start = 2
231+ return mkList(xs[0 ..< start] & r.fnTail(ns, xs[start .. ^1], scope))
232+ of "defn", "defn-", "defmacro":
233+ if xs.len < 3: fail(h & " requires a name and parameters")
234+ xs[1] = ns.define(xs[1], h == "defn-")
235+ var start = 2
236+ if start < xs.len and xs[start].kind == kStr: inc start
237+ if start < xs.len and xs[start].kind == kMap: inc start
238+ var scope = locals
239+ if h == "defmacro":
240+ scope.incl "&form"
241+ scope.incl "&env"
242+ return mkList(xs[0 ..< start] & r.fnTail(ns, xs[start .. ^1], scope))
243+ of "def", "declare":
244+ if xs.len < 2: fail(h & " requires a name")
245+ let stop = if h == "declare": xs.len else: 2
246+ for i in 1 ..< stop:
247+ xs[i] = ns.define(xs[i])
248+ for i in stop ..< xs.len: xs[i] = r.walk(ns, xs[i], locals)
249+ return mkList(xs)
250+ of "let", "let*", "loop", "loop*", "when-let", "if-let", "doseq", "dotimes":
251+ if xs.len < 2 or xs[1].kind != kVector: fail(h & " requires bindings")
252+ var bs = xs[1].items
253+ if bs.len mod 2 != 0: fail(h & " requires paired bindings")
254+ if h in ["when-let", "if-let", "doseq", "dotimes"] and bs.len != 2:
255+ fail(h & " supports exactly one binding")
256+ var scope = locals
257+ var i = 0
258+ while i < bs.len:
259+ bs[i + 1] = r.walk(ns, bs[i + 1], scope)
260+ let original = bs[i]
261+ bs[i] = r.pattern(ns, original, scope)
262+ bindNames(original, scope)
263+ i += 2
264+ xs[1] = mkVector(bs)
265+ for j in 2 ..< xs.len:
266+ xs[j] = r.walk(ns, xs[j], if h == "if-let" and j >= 3: locals else: scope)
267+ return mkList(xs)
268+ of "catch":
269+ if xs.len < 3: fail("catch requires a class and binding")
270+ var scope = locals
271+ scope.incl simple(xs[2], "catch binding")
272+ for i in 3 ..< xs.len: xs[i] = r.walk(ns, xs[i], scope)
273+ return mkList(xs)
274+ of "->", "->>":
275+ # Expand before resolving: a bare step can be a syntax head (e.g. do).
276+ if xs.len < 2: fail(h & " requires an expression")
277+ var acc = xs[1]
278+ for i in 2 ..< xs.len:
279+ let step = xs[i]
280+ if step.kind == kList and step.items.len > 0:
281+ let parts = step.items
282+ if h == "->": acc = mkList(@[parts[0], acc] & parts[1 .. ^1])
283+ else: acc = mkList(parts & @[acc])
284+ else: acc = mkList(@[step, acc])
285+ return r.walk(ns, acc, locals)
286+ else: discard
287+ for i in 0 ..< xs.len:
288+ if i == 0 and h in syntaxHeads: continue
289+ if h == "cond" and i mod 2 == 1 and isSym(xs[i], "else"): continue
290+ xs[i] = r.walk(ns, xs[i], locals)
291+ if h.len == 0 and xs[0].kind == kSymbol and xs[0].s in locals and
292+ (xs[0].s in syntaxHeads or xs[0].s in namespaceHeads):
293+ # The compiler dispatches by head spelling before consulting its locals.
294+ # An expression head keeps this a value call without renaming bindings.
295+ xs[0] = mkList(@[mkSymbol("do"), xs[0]])
296+ mkList(xs)
297+
298+proc process(r: Resolver, forms: seq[Value], expected = "")
299+proc load(r: Resolver, name: string) =
300+ validNamespace(name)
301+ if name == "clojure.core": return
302+ if name in r.active: fail("dependency cycle: " & (r.active & @[name]).join(" -> "))
303+ if name in r.loaded: return
304+ let relative = name.replace('.', '/').replace('-', '_')
305+ var path = ""
306+ for root in r.roots:
307+ for ext in [".clj", ".cljc"]:
308+ let candidate = root / (relative & ext)
309+ if fileExists(candidate):
310+ path = candidate
311+ break
312+ if path.len > 0: break
313+ if path.len == 0: fail("cannot find " & name & " in source roots: " & r.roots.join(", "))
314+ try:
315+ r.process(readAll(readFile(path)), name)
316+ except IOError as e:
317+ fail("cannot read " & path & ": " & e.msg)
318+
319+proc requireSpec(r: Resolver, ns: Namespace, spec: Value) =
320+ var xs: seq[Value]
321+ if spec.kind == kSymbol: xs = @[spec]
322+ elif spec.kind == kVector: xs = spec.items
323+ else: fail("require expects a symbol or vector libspec")
324+ if xs.len == 0: fail("empty require libspec")
325+ let name = simple(xs[0], "required namespace")
326+ validNamespace(name)
327+ var alias = ""
328+ var refers: seq[string]
329+ var seen: HashSet[string]
330+ var i = 1
331+ while i < xs.len:
332+ if i + 1 >= xs.len or xs[i].kind != kKeyword: fail("invalid require options")
333+ let option = xs[i].s
334+ if option in seen: fail("duplicate require option: " & option)
335+ seen.incl option
336+ case option
337+ of "as": alias = simple(xs[i + 1], "alias")
338+ of "refer":
339+ if xs[i + 1].kind != kVector: fail(":refer supports only an explicit vector")
340+ for x in xs[i + 1].items: refers.add simple(x, "referred name")
341+ else: fail("unsupported require option: :" & option)
342+ i += 2
343+ r.load(name)
344+ ns.required.incl name
345+ if alias.len > 0:
346+ if ns.aliases.hasKey(alias) and ns.aliases[alias] != name:
347+ fail("conflicting alias: " & alias)
348+ ns.aliases[alias] = name
349+ for local in refers:
350+ r.checkVar(ns, name, local, allowSyntax = true)
351+ let target = name & "/" & local
352+ if local in ns.defs or (ns.refers.hasKey(local) and ns.refers[local] != target):
353+ fail("conflicting referred name: " & local)
354+ ns.refers[local] = target
355+
356+proc flatten(forms: seq[Value]): seq[Value] =
357+ for f in forms:
358+ if headName(f) == "do": result.add flatten(f.items[1 .. ^1])
359+ else: result.add f
360+
361+proc process(r: Resolver, forms: seq[Value], expected = "") =
362+ let flat = flatten(forms)
363+ var name = "user"
364+ var declaration = -1
365+ for i, f in flat:
366+ if headName(f) == "ns":
367+ if declaration >= 0 or i != 0: fail("ns must be the first form and occur once per source")
368+ if f.items.len < 2: fail("ns requires a name")
369+ name = simple(f.items[1], "namespace name")
370+ validNamespace(name)
371+ declaration = i
372+ if expected.len > 0 and (declaration < 0 or name != expected):
373+ fail("source for " & expected & " must declare (ns " & expected & ")")
374+ if name in r.active: fail("dependency cycle: " & (r.active & @[name]).join(" -> "))
375+ let ns = r.space(name)
376+
377+ r.active.add name
378+ try:
379+ for f in flat:
380+ case ns.syntaxHead(f, initHashSet[string]())
381+ of "ns":
382+ let xs = f.items
383+ var start = 2
384+ if start < xs.len and xs[start].kind == kStr: inc start
385+ if start < xs.len and xs[start].kind == kMap: inc start
386+ for i in start ..< xs.len:
387+ let clause = xs[i]
388+ if clause.kind != kList or clause.items.len == 0 or
389+ clause.items[0].kind != kKeyword or clause.items[0].s != "require":
390+ fail("only :require clauses are supported in ns")
391+ for spec in clause.items[1 .. ^1]: r.requireSpec(ns, spec)
392+ of "require":
393+ if f.items.len < 2: fail("require expects quoted libspecs")
394+ for arg in f.items[1 .. ^1]:
395+ if headName(arg) != "quote" or arg.items.len != 2:
396+ fail("dynamic require is unsupported; quote each libspec")
397+ r.requireSpec(ns, arg.items[1])
398+ else:
399+ r.output.add r.walk(ns, f, initHashSet[string]())
400+ r.loaded.incl name
401+ finally:
402+ discard r.active.pop()
403+
404+proc resolveSource*(src: string, sourceRoots: seq[string]): seq[Value] =
405+ ## Read source text and splice dependencies at their first require, once per
406+ ## call. Roots are searched in order, .clj before .cljc. Missing/mismatched
407+ ## namespaces, cycles, unsupported imports and dynamic requires raise CljError.
408+ ## No namespace state or load cache survives a call. Definitions are visible
409+ ## from their declaration onward (including their own initializer/body).
410+ ## Forward references require declare. Qualified references require a direct
411+ ## require, except for the current namespace and implicit clojure.core.
412+ ## Unknown and externally private vars are rejected during analysis.
413+ let r = Resolver(roots: sourceRoots, cores: registeredCoreNames())
414+ r.process(readAll(src))
415+ r.output
added stdlib/clonim/core.clj +4 -0
new file mode 100644
@@ -0,0 +1,4 @@
1+(ns clonim.core)
2+
3+;; Host-dependent operations stay behind a small runtime primitive.
4+(defn now-ms [] (*epoch-time-ms*))
new file mode 100644
@@ -0,0 +1,4 @@
1+(ns clonim.core)
2+
3+;; Host-dependent operations stay behind a small runtime primitive.
4+(defn now-ms [] (*epoch-time-ms*))
modified tests/rebinding.expected +1 -1
@@ -1,5 +1,5 @@
11 11
22 1000
33 10
4-0
4+10
55 12
@@ -1,5 +1,5 @@
1 111 11
2 10002 1000
3 103 10
4-04+10
5 125 12
added tests/test_namespaces.nim +217 -0
new file mode 100644
@@ -0,0 +1,217 @@
1+## Run with: nim r --hints:off --path:src tests/test_namespaces.nim
2+import std/[unittest, os, tempfiles, strutils, tables]
3+import runtime, reader, core, namespaces
4+
5+proc checkForms(src, expected: string, roots: seq[string] = @[]) =
6+ let actual = resolveSource(src, roots)
7+ let wanted = readAll(expected)
8+ check actual.len == wanted.len
9+ if actual.len == wanted.len:
10+ for i in 0 ..< actual.len:
11+ checkpoint "actual: " & prStr(actual[i]) & "; expected: " & prStr(wanted[i])
12+ check equals(actual[i], wanted[i])
13+
14+proc rejects(src, message: string, roots: seq[string] = @[]) =
15+ var rejected = false
16+ try:
17+ discard resolveSource(src, roots)
18+ except CljError as e:
19+ rejected = true
20+ checkpoint e.msg
21+ check message in e.msg
22+ check rejected
23+
24+suite "static namespace resolution":
25+ test "definitions resolve sequentially, including core shadowing":
26+ checkForms("(inc 1) (def inc (fn [x] x)) (inc 2)",
27+ "(clojure.core/inc 1) (def user/inc (fn [x] x)) (user/inc 2)")
28+ checkForms("(defn before [x] (inc x)) (def inc (fn [x] x))",
29+ "(defn user/before [x] (clojure.core/inc x)) (def user/inc (fn [x] x))")
30+ checkForms("(when true 1) (def when (fn [x] x)) (when 2)",
31+ "(when true 1) (def user/when (fn [x] x)) (user/when 2)")
32+
33+ test "unknown symbols and forward references require declare":
34+ rejects("missing", "unable to resolve symbol missing")
35+ rejects("(defn first [] (later)) (defn later [] 1)", "unable to resolve symbol later")
36+ checkForms("(declare later) (defn first [] (later)) (defn later [] 1)",
37+ "(declare user/later) (defn user/first [] (user/later)) (defn user/later [] 1)")
38+
39+ test "definitions are visible in their own bodies":
40+ checkForms("(defn f [x] (f x)) (defn- hidden [] (hidden))",
41+ "(defn user/f [x] (user/f x)) (defn- user/hidden [] (user/hidden))")
42+ checkForms("(def f (fn [] f))", "(def user/f (fn [] user/f))")
43+
44+ test "locals shadow macros, including comment and binding macros":
45+ checkForms("(let [when +] (when 1 2))",
46+ "(let [when clojure.core/+] ((do when) 1 2))")
47+ checkForms("(fn [comment let -> require] [(comment 1) (let 2) (-> 3) (require 4)])",
48+ "(fn [comment let -> require] [((do comment) 1) ((do let) 2) ((do ->) 3) ((do require) 4)])")
49+ checkForms("(fn when [x] (when x))", "(fn when [x] ((do when) x))")
50+ checkForms("(let [when +] (clojure.core/when true (when 1 2)))",
51+ "(let [when clojure.core/+] (when true ((do when) 1 2)))")
52+
53+ test "true special forms are not shadowed in operator position":
54+ checkForms("(let [if +] (if true 1 2))", "(let [if clojure.core/+] (if true 1 2))")
55+
56+ test "ordinary locals, sequential bindings, functions and destructuring":
57+ checkForms("(let [x 1 y x] (fn self ([z] (self z y)) ([z q] (+ x z q))))",
58+ "(let [x 1 y x] (fn self ([z] (self z y)) ([z q] (clojure.core/+ x z q))))")
59+ checkForms("(def fallback 9) (let [{:keys [x] :or {x fallback} :as m} {}] [x m])",
60+ "(def user/fallback 9) (let [{:keys [x] :or {x user/fallback} :as m} {}] [x m])")
61+ checkForms("(loop [[x & more :as all] [1 2]] (recur all))",
62+ "(loop [[x & more :as all] [1 2]] (recur all))")
63+ rejects("(let [x y y 1] x)", "unable to resolve symbol y")
64+
65+ test "conditional bindings and catch scope do not leak":
66+ rejects("(if-let [x 1] x x)", "unable to resolve symbol x")
67+ checkForms("(def x 2) (if-let [x 1] x x)",
68+ "(def user/x 2) (if-let [x 1] x user/x)")
69+ checkForms("(try 1 (catch Exception e (str e)) (finally 2))",
70+ "(try 1 (catch Exception e (clojure.core/str e)) (finally 2))")
71+ rejects("(try 1 (catch Exception e e) (finally e))", "unable to resolve symbol e")
72+
73+ test "quotes and comments are data and do not declare vars":
74+ checkForms("'(missing missing.ns/x (def imaginary 1)) (comment missing)",
75+ "(quote (missing missing.ns/x (def imaginary 1))) (comment missing)")
76+ rejects("'(def imaginary 1) imaginary", "unable to resolve symbol imaginary")
77+ rejects("(comment (def imaginary 1)) imaginary", "unable to resolve symbol imaginary")
78+
79+ test "macro bindings and threading keep their scopes":
80+ checkForms("(defmacro m [x] (list 'quote x &form &env))",
81+ "(defmacro user/m [x] (clojure.core/list (quote quote) x &form &env))")
82+ checkForms("(let [when +] (-> 1 (when 2)))",
83+ "(let [when clojure.core/+] ((do when) 1 2))")
84+
85+ test "qualified core and current namespace names are checked":
86+ checkForms("(def x 1) user/x (clojure.core// 6 2)",
87+ "(def user/x 1) user/x (clojure.core// 6 2)")
88+ rejects("user/nope", "no definition user/nope")
89+ rejects("clojure.core/nope", "no definition clojure.core/nope")
90+ rejects("stranger/x", "namespace stranger is not required")
91+
92+ test "dynamic namespace operations and unsupported imports fail":
93+ for src in ["(require variable)", "(apply require [])", "(fn [] (require 'foo))",
94+ "(import 'Thing)", "(ns app (:import Thing))", "(require '[foo :refer :all])"]:
95+ rejects(src, "Namespace error")
96+
97+ test "core discovery preserves caller cells even after analysis failure":
98+ let saved = globals
99+ globals = initTable[string, VarCell]()
100+ try:
101+ registerCore()
102+ discard setVar("inc", mkInt(42))
103+ let cell = varCell("inc")
104+ checkForms("(inc 1)", "(clojure.core/inc 1)")
105+ rejects("missing", "unable to resolve")
106+ check varCell("inc") == cell
107+ check getVar("inc").i == 42
108+ registerNamespaceCore()
109+ check varCell("clojure.core/inc") == cell
110+ finally:
111+ globals = saved
112+
113+suite "namespace source loader":
114+ var root: string
115+ setup:
116+ root = createTempDir("clonim-namespaces-", "")
117+ createDir(root / "sample")
118+ writeFile(root / "sample" / "lib_one.clj", """
119+ (ns sample.lib-one)
120+ (def value 7)
121+ (defn- hidden [] value)
122+ (defn public [] (hidden))
123+ (def when (fn [x] x))
124+ """)
125+ writeFile(root / "consumer.clj", """
126+ (ns consumer (:require [sample.lib-one :as inner :refer [value]]))
127+ (def result inner/value)
128+ """)
129+ writeFile(root / "other.clj", """
130+ (ns other (:require sample.lib-one))
131+ (def result sample.lib-one/value)
132+ """)
133+ teardown:
134+ removeDir(root)
135+
136+ test "alias, refer and plain require produce the same canonical var":
137+ let forms = resolveSource("""
138+ (ns app (:require [sample.lib-one :as lib :refer [value]]))
139+ (require 'sample.lib-one)
140+ [lib/value value sample.lib-one/value]
141+ """, @[root])
142+ check forms.len == 5
143+ check prStr(forms[^1]) == "[sample.lib-one/value sample.lib-one/value sample.lib-one/value]"
144+ check prStr(forms[2]) == "(defn sample.lib-one/public [] (sample.lib-one/hidden))"
145+
146+ test "plain require imports no unqualified names or aliases":
147+ rejects("(require 'sample.lib-one) value", "unable to resolve symbol value", @[root])
148+ rejects("(require 'sample.lib-one) lib/value", "namespace lib is not required", @[root])
149+ let forms = resolveSource("(require 'sample.lib-one) sample.lib-one/value", @[root])
150+ check prStr(forms[^1]) == "sample.lib-one/value"
151+
152+ test "alias require imports no unqualified names":
153+ rejects("(require '[sample.lib-one :as lib]) value", "unable to resolve symbol value", @[root])
154+
155+ test "aliases and refers do not leak from dependencies":
156+ rejects("(require 'consumer) inner/value", "namespace inner is not required", @[root])
157+ rejects("(require 'consumer) value", "unable to resolve symbol value", @[root])
158+ rejects("(require 'consumer) sample.lib-one/value", "namespace sample.lib-one is not required", @[root])
159+ rejects("(require '[sample.lib-one :as lib]) (require 'consumer) inner/value",
160+ "namespace inner is not required", @[root])
161+
162+ test "required missing vars and private vars fail through all spellings":
163+ for member in ["missing", "hidden"]:
164+ let reason = if member == "hidden": "private var sample.lib-one/hidden" else: "no definition sample.lib-one/missing"
165+ rejects("(require 'sample.lib-one) sample.lib-one/" & member, reason, @[root])
166+ rejects("(require '[sample.lib-one :as lib]) lib/" & member, reason, @[root])
167+ rejects("(require '[sample.lib-one :refer [" & member & "]])", reason, @[root])
168+
169+ test "referred and aliased vars can shadow core macro names":
170+ let forms = resolveSource("(require '[sample.lib-one :as lib :refer [when]]) (when 1) (lib/when 2)", @[root])
171+ check prStr(forms[^2]) == "(sample.lib-one/when 1)"
172+ check prStr(forms[^1]) == "(sample.lib-one/when 2)"
173+
174+ test "core aliases and explicit core syntax referrals":
175+ checkForms("(ns app (:require [clojure.core :as c :refer [when]])) (c/when true (c/inc 1)) (when true 2)",
176+ "(when true (clojure.core/inc 1)) (when true 2)")
177+
178+ test "conflicting aliases, referrals and definitions are rejected":
179+ rejects("(require '[sample.lib-one :as lib] '[consumer :as lib])", "conflicting alias", @[root])
180+ rejects("(require '[consumer :refer [result]] '[other :refer [result]])", "conflicting referred name", @[root])
181+ rejects("(require '[sample.lib-one :refer [value]]) (def value 1)", "definition conflicts", @[root])
182+ rejects("(def value 1) (require '[sample.lib-one :refer [value]])", "conflicting referred name", @[root])
183+
184+ test "diamond dependencies are loaded once in dependency order":
185+ let forms = resolveSource("(require 'consumer 'other 'sample.lib-one 'consumer)", @[root])
186+ check forms.len == 6
187+ check prStr(forms[0]) == "(def sample.lib-one/value 7)"
188+ check prStr(forms[^2]) == "(def consumer/result sample.lib-one/value)"
189+ check prStr(forms[^1]) == "(def other/result sample.lib-one/value)"
190+ # A new resolver call has its own load cache.
191+ check resolveSource("(require 'consumer 'other)", @[root]).len == 6
192+ rejects("consumer/result", "namespace consumer is not required", @[root])
193+
194+ test "missing source and mismatched declarations are rejected":
195+ rejects("(require 'absent)", "cannot find absent", @[root])
196+ writeFile(root / "wrong.clj", "(ns different)")
197+ rejects("(require 'wrong)", "must declare (ns wrong)", @[root])
198+ writeFile(root / "bare.clj", "(def x 1)")
199+ rejects("(require 'bare)", "must declare (ns bare)", @[root])
200+
201+ test "cycles report the dependency chain including the root":
202+ writeFile(root / "a.clj", "(ns a (:require b))")
203+ writeFile(root / "b.clj", "(ns b (:require a))")
204+ rejects("(require 'a)", "user -> a -> b -> a", @[root])
205+ rejects("(ns a (:require b))", "a -> b -> a", @[root])
206+
207+ test "definitions cannot see requires occurring later":
208+ rejects("sample.lib-one/value (require 'sample.lib-one)",
209+ "namespace sample.lib-one is not required", @[root])
210+
211+ test "clj is preferred to cljc and roots retain their order":
212+ writeFile(root / "choice.clj", "(ns choice) (def value 1)")
213+ writeFile(root / "choice.cljc", "(ns choice) (def value 2)")
214+ createDir(root / "first")
215+ writeFile(root / "first" / "choice.cljc", "(ns choice) (def value 3)")
216+ checkForms("(require 'choice)", "(def choice/value 1)", @[root])
217+ checkForms("(require 'choice)", "(def choice/value 3)", @[root / "first", root])
new file mode 100644
@@ -0,0 +1,217 @@
1+## Run with: nim r --hints:off --path:src tests/test_namespaces.nim
2+import std/[unittest, os, tempfiles, strutils, tables]
3+import runtime, reader, core, namespaces
4+
5+proc checkForms(src, expected: string, roots: seq[string] = @[]) =
6+ let actual = resolveSource(src, roots)
7+ let wanted = readAll(expected)
8+ check actual.len == wanted.len
9+ if actual.len == wanted.len:
10+ for i in 0 ..< actual.len:
11+ checkpoint "actual: " & prStr(actual[i]) & "; expected: " & prStr(wanted[i])
12+ check equals(actual[i], wanted[i])
13+
14+proc rejects(src, message: string, roots: seq[string] = @[]) =
15+ var rejected = false
16+ try:
17+ discard resolveSource(src, roots)
18+ except CljError as e:
19+ rejected = true
20+ checkpoint e.msg
21+ check message in e.msg
22+ check rejected
23+
24+suite "static namespace resolution":
25+ test "definitions resolve sequentially, including core shadowing":
26+ checkForms("(inc 1) (def inc (fn [x] x)) (inc 2)",
27+ "(clojure.core/inc 1) (def user/inc (fn [x] x)) (user/inc 2)")
28+ checkForms("(defn before [x] (inc x)) (def inc (fn [x] x))",
29+ "(defn user/before [x] (clojure.core/inc x)) (def user/inc (fn [x] x))")
30+ checkForms("(when true 1) (def when (fn [x] x)) (when 2)",
31+ "(when true 1) (def user/when (fn [x] x)) (user/when 2)")
32+
33+ test "unknown symbols and forward references require declare":
34+ rejects("missing", "unable to resolve symbol missing")
35+ rejects("(defn first [] (later)) (defn later [] 1)", "unable to resolve symbol later")
36+ checkForms("(declare later) (defn first [] (later)) (defn later [] 1)",
37+ "(declare user/later) (defn user/first [] (user/later)) (defn user/later [] 1)")
38+
39+ test "definitions are visible in their own bodies":
40+ checkForms("(defn f [x] (f x)) (defn- hidden [] (hidden))",
41+ "(defn user/f [x] (user/f x)) (defn- user/hidden [] (user/hidden))")
42+ checkForms("(def f (fn [] f))", "(def user/f (fn [] user/f))")
43+
44+ test "locals shadow macros, including comment and binding macros":
45+ checkForms("(let [when +] (when 1 2))",
46+ "(let [when clojure.core/+] ((do when) 1 2))")
47+ checkForms("(fn [comment let -> require] [(comment 1) (let 2) (-> 3) (require 4)])",
48+ "(fn [comment let -> require] [((do comment) 1) ((do let) 2) ((do ->) 3) ((do require) 4)])")
49+ checkForms("(fn when [x] (when x))", "(fn when [x] ((do when) x))")
50+ checkForms("(let [when +] (clojure.core/when true (when 1 2)))",
51+ "(let [when clojure.core/+] (when true ((do when) 1 2)))")
52+
53+ test "true special forms are not shadowed in operator position":
54+ checkForms("(let [if +] (if true 1 2))", "(let [if clojure.core/+] (if true 1 2))")
55+
56+ test "ordinary locals, sequential bindings, functions and destructuring":
57+ checkForms("(let [x 1 y x] (fn self ([z] (self z y)) ([z q] (+ x z q))))",
58+ "(let [x 1 y x] (fn self ([z] (self z y)) ([z q] (clojure.core/+ x z q))))")
59+ checkForms("(def fallback 9) (let [{:keys [x] :or {x fallback} :as m} {}] [x m])",
60+ "(def user/fallback 9) (let [{:keys [x] :or {x user/fallback} :as m} {}] [x m])")
61+ checkForms("(loop [[x & more :as all] [1 2]] (recur all))",
62+ "(loop [[x & more :as all] [1 2]] (recur all))")
63+ rejects("(let [x y y 1] x)", "unable to resolve symbol y")
64+
65+ test "conditional bindings and catch scope do not leak":
66+ rejects("(if-let [x 1] x x)", "unable to resolve symbol x")
67+ checkForms("(def x 2) (if-let [x 1] x x)",
68+ "(def user/x 2) (if-let [x 1] x user/x)")
69+ checkForms("(try 1 (catch Exception e (str e)) (finally 2))",
70+ "(try 1 (catch Exception e (clojure.core/str e)) (finally 2))")
71+ rejects("(try 1 (catch Exception e e) (finally e))", "unable to resolve symbol e")
72+
73+ test "quotes and comments are data and do not declare vars":
74+ checkForms("'(missing missing.ns/x (def imaginary 1)) (comment missing)",
75+ "(quote (missing missing.ns/x (def imaginary 1))) (comment missing)")
76+ rejects("'(def imaginary 1) imaginary", "unable to resolve symbol imaginary")
77+ rejects("(comment (def imaginary 1)) imaginary", "unable to resolve symbol imaginary")
78+
79+ test "macro bindings and threading keep their scopes":
80+ checkForms("(defmacro m [x] (list 'quote x &form &env))",
81+ "(defmacro user/m [x] (clojure.core/list (quote quote) x &form &env))")
82+ checkForms("(let [when +] (-> 1 (when 2)))",
83+ "(let [when clojure.core/+] ((do when) 1 2))")
84+
85+ test "qualified core and current namespace names are checked":
86+ checkForms("(def x 1) user/x (clojure.core// 6 2)",
87+ "(def user/x 1) user/x (clojure.core// 6 2)")
88+ rejects("user/nope", "no definition user/nope")
89+ rejects("clojure.core/nope", "no definition clojure.core/nope")
90+ rejects("stranger/x", "namespace stranger is not required")
91+
92+ test "dynamic namespace operations and unsupported imports fail":
93+ for src in ["(require variable)", "(apply require [])", "(fn [] (require 'foo))",
94+ "(import 'Thing)", "(ns app (:import Thing))", "(require '[foo :refer :all])"]:
95+ rejects(src, "Namespace error")
96+
97+ test "core discovery preserves caller cells even after analysis failure":
98+ let saved = globals
99+ globals = initTable[string, VarCell]()
100+ try:
101+ registerCore()
102+ discard setVar("inc", mkInt(42))
103+ let cell = varCell("inc")
104+ checkForms("(inc 1)", "(clojure.core/inc 1)")
105+ rejects("missing", "unable to resolve")
106+ check varCell("inc") == cell
107+ check getVar("inc").i == 42
108+ registerNamespaceCore()
109+ check varCell("clojure.core/inc") == cell
110+ finally:
111+ globals = saved
112+
113+suite "namespace source loader":
114+ var root: string
115+ setup:
116+ root = createTempDir("clonim-namespaces-", "")
117+ createDir(root / "sample")
118+ writeFile(root / "sample" / "lib_one.clj", """
119+ (ns sample.lib-one)
120+ (def value 7)
121+ (defn- hidden [] value)
122+ (defn public [] (hidden))
123+ (def when (fn [x] x))
124+ """)
125+ writeFile(root / "consumer.clj", """
126+ (ns consumer (:require [sample.lib-one :as inner :refer [value]]))
127+ (def result inner/value)
128+ """)
129+ writeFile(root / "other.clj", """
130+ (ns other (:require sample.lib-one))
131+ (def result sample.lib-one/value)
132+ """)
133+ teardown:
134+ removeDir(root)
135+
136+ test "alias, refer and plain require produce the same canonical var":
137+ let forms = resolveSource("""
138+ (ns app (:require [sample.lib-one :as lib :refer [value]]))
139+ (require 'sample.lib-one)
140+ [lib/value value sample.lib-one/value]
141+ """, @[root])
142+ check forms.len == 5
143+ check prStr(forms[^1]) == "[sample.lib-one/value sample.lib-one/value sample.lib-one/value]"
144+ check prStr(forms[2]) == "(defn sample.lib-one/public [] (sample.lib-one/hidden))"
145+
146+ test "plain require imports no unqualified names or aliases":
147+ rejects("(require 'sample.lib-one) value", "unable to resolve symbol value", @[root])
148+ rejects("(require 'sample.lib-one) lib/value", "namespace lib is not required", @[root])
149+ let forms = resolveSource("(require 'sample.lib-one) sample.lib-one/value", @[root])
150+ check prStr(forms[^1]) == "sample.lib-one/value"
151+
152+ test "alias require imports no unqualified names":
153+ rejects("(require '[sample.lib-one :as lib]) value", "unable to resolve symbol value", @[root])
154+
155+ test "aliases and refers do not leak from dependencies":
156+ rejects("(require 'consumer) inner/value", "namespace inner is not required", @[root])
157+ rejects("(require 'consumer) value", "unable to resolve symbol value", @[root])
158+ rejects("(require 'consumer) sample.lib-one/value", "namespace sample.lib-one is not required", @[root])
159+ rejects("(require '[sample.lib-one :as lib]) (require 'consumer) inner/value",
160+ "namespace inner is not required", @[root])
161+
162+ test "required missing vars and private vars fail through all spellings":
163+ for member in ["missing", "hidden"]:
164+ let reason = if member == "hidden": "private var sample.lib-one/hidden" else: "no definition sample.lib-one/missing"
165+ rejects("(require 'sample.lib-one) sample.lib-one/" & member, reason, @[root])
166+ rejects("(require '[sample.lib-one :as lib]) lib/" & member, reason, @[root])
167+ rejects("(require '[sample.lib-one :refer [" & member & "]])", reason, @[root])
168+
169+ test "referred and aliased vars can shadow core macro names":
170+ let forms = resolveSource("(require '[sample.lib-one :as lib :refer [when]]) (when 1) (lib/when 2)", @[root])
171+ check prStr(forms[^2]) == "(sample.lib-one/when 1)"
172+ check prStr(forms[^1]) == "(sample.lib-one/when 2)"
173+
174+ test "core aliases and explicit core syntax referrals":
175+ checkForms("(ns app (:require [clojure.core :as c :refer [when]])) (c/when true (c/inc 1)) (when true 2)",
176+ "(when true (clojure.core/inc 1)) (when true 2)")
177+
178+ test "conflicting aliases, referrals and definitions are rejected":
179+ rejects("(require '[sample.lib-one :as lib] '[consumer :as lib])", "conflicting alias", @[root])
180+ rejects("(require '[consumer :refer [result]] '[other :refer [result]])", "conflicting referred name", @[root])
181+ rejects("(require '[sample.lib-one :refer [value]]) (def value 1)", "definition conflicts", @[root])
182+ rejects("(def value 1) (require '[sample.lib-one :refer [value]])", "conflicting referred name", @[root])
183+
184+ test "diamond dependencies are loaded once in dependency order":
185+ let forms = resolveSource("(require 'consumer 'other 'sample.lib-one 'consumer)", @[root])
186+ check forms.len == 6
187+ check prStr(forms[0]) == "(def sample.lib-one/value 7)"
188+ check prStr(forms[^2]) == "(def consumer/result sample.lib-one/value)"
189+ check prStr(forms[^1]) == "(def other/result sample.lib-one/value)"
190+ # A new resolver call has its own load cache.
191+ check resolveSource("(require 'consumer 'other)", @[root]).len == 6
192+ rejects("consumer/result", "namespace consumer is not required", @[root])
193+
194+ test "missing source and mismatched declarations are rejected":
195+ rejects("(require 'absent)", "cannot find absent", @[root])
196+ writeFile(root / "wrong.clj", "(ns different)")
197+ rejects("(require 'wrong)", "must declare (ns wrong)", @[root])
198+ writeFile(root / "bare.clj", "(def x 1)")
199+ rejects("(require 'bare)", "must declare (ns bare)", @[root])
200+
201+ test "cycles report the dependency chain including the root":
202+ writeFile(root / "a.clj", "(ns a (:require b))")
203+ writeFile(root / "b.clj", "(ns b (:require a))")
204+ rejects("(require 'a)", "user -> a -> b -> a", @[root])
205+ rejects("(ns a (:require b))", "a -> b -> a", @[root])
206+
207+ test "definitions cannot see requires occurring later":
208+ rejects("sample.lib-one/value (require 'sample.lib-one)",
209+ "namespace sample.lib-one is not required", @[root])
210+
211+ test "clj is preferred to cljc and roots retain their order":
212+ writeFile(root / "choice.clj", "(ns choice) (def value 1)")
213+ writeFile(root / "choice.cljc", "(ns choice) (def value 2)")
214+ createDir(root / "first")
215+ writeFile(root / "first" / "choice.cljc", "(ns choice) (def value 3)")
216+ checkForms("(require 'choice)", "(def choice/value 1)", @[root])
217+ checkForms("(require 'choice)", "(def choice/value 3)", @[root / "first", root])