nandi/clonimpublic Fork 0
430bb9f
Commits
Clone
git clone https://git.rickub.com/nandi/clonim.git
git clone ssh://git@rickub.com/nandi/clonim.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

perf: port the codegen and int-specialisation work onto lazy seqs

Rebasing the perf branch onto main meant choosing what still applies now that
collections are persistent and seqs are lazy. Measured rather than guessed, and
the answer was clean: every win that mattered lives in the compiler and is
independent of how values are represented.

  * expression codegen — forms that need no statements compile to a Nim
    expression instead of a fresh `var t: Value = NilV` slot per subexpression.
  * direct calls — each fixed-arity fn clause gets a real Nim proc taking its
    params positionally, and call sites that know the arity skip the dispatcher.
  * inlined primitives — two-argument arithmetic and comparison builtins get
    inline forms emitted at the call site.
  * int specialisation — loop variables and fn parameters that are provably
    integral are held as raw int64, so a numeric loop never builds a Value at
    all. This is where the bulk of the speedup is.

Soundness comes from a whole-program scan, not a runtime check: def and defn
are the only paths to setVar and there is no eval, defmacro or intern, so a
name no def form targets keeps its registerCore value for the process. Where
that holds, `+` is arithmetic and no guard is needed; where it does not, call
sites carry a cellIs check and nothing specialises.

Deliberately NOT ported: the unboxed Value representation, which conflicts with
PVec/PMap and with force() memoising through the ref, and the eager sequence
library, which laziness replaces. The first turns out to be nearly redundant --
a specialised loop holds int64 in registers regardless of what Value is -- and
the second no longer describes this runtime.

Two bugs the ported tests caught, both pre-existing on main:

  * Cursor.next walked `tl` as a node, so as soon as a tail bottomed out in a
    concrete collection the seq ended after its head. `(cons 3 '(3 4))` read as
    `(3)`, and any reduce over a cons'd list silently lost its tail. The cursor
    now switches to indexing when the chain reaches a real collection.
  * quot and rem let Nim's `div` raise an uncatchable defect on a zero divisor,
    where `/` had always reported a catchable error. Both go through checked
    helpers now, as does mod.

Best-of-N round-robin, compute excluding startup, against main:

                        main    this
  fib 30               756 ms   3 ms
  loop 10M            2454 ms   3 ms
  seqs                 869 ms  853 ms
  persistent-bench     123 ms  120 ms
  lazy-bench             0 ms    0 ms

seqs barely moves because it is now bounded by laziness, not by codegen: a cons
cell and a memoising thunk per element make `(count (range 1000000))` 299 ms
where the eager version was 23 ms. That is the price of infinite seqs and early
exit, and the standard answer is chunking rather than anything in this commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nandithebull committed 2026-09-20T18:57:15-07:00 Browse files
430bb9f parent: 4de7343
modified .gitignore +1 -0
@@ -2,3 +2,4 @@ bin/
22 nimcache/
33 .clj-kondo/.cache/
44 .lsp/
5+bench/bin/
@@ -2,3 +2,4 @@ bin/
2 nimcache/2 nimcache/
3 .clj-kondo/.cache/3 .clj-kondo/.cache/
4 .lsp/4 .lsp/
5+bench/bin/
added bench/fib.clj +2 -0
new file mode 100644
@@ -0,0 +1,2 @@
1+(defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))
2+(println (fib 30))
new file mode 100644
@@ -0,0 +1,2 @@
1+(defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))
2+(println (fib 30))
added bench/hello.clj +1 -0
new file mode 100644
@@ -0,0 +1 @@
1+(println "hello")
new file mode 100644
@@ -0,0 +1 @@
1+(println "hello")
added bench/loop.clj +4 -0
new file mode 100644
@@ -0,0 +1,4 @@
1+(defn sum-to [n]
2+ (loop [i 0 acc 0]
3+ (if (< i n) (recur (+ i 1) (+ acc i)) acc)))
4+(println (sum-to 10000000))
new file mode 100644
@@ -0,0 +1,4 @@
1+(defn sum-to [n]
2+ (loop [i 0 acc 0]
3+ (if (< i n) (recur (+ i 1) (+ acc i)) acc)))
4+(println (sum-to 10000000))
added bench/measure.py +54 -0
new file mode 100755
@@ -0,0 +1,54 @@
1+#!/usr/bin/env python3
2+"""Time clonim's benchmark binaries, optionally against jolt.
3+
4+Two things make these numbers trustworthy rather than merely small:
5+
6+ * best-of-N, not mean. A benchmark process contends with whatever else the
7+ machine is doing; contention only ever adds time, so the minimum is the
8+ cleanest estimate of the work itself.
9+ * round-robin, not grouped. Running every binary once per round, rather than
10+ one binary N times before moving on, means a slow stretch of machine time
11+ lands on all of them instead of whichever happened to be running.
12+
13+Compute is reported as total minus the `hello` time for the same toolchain,
14+which is how a 0.5 ms native start is kept from flattering clonim against a
15+runtime that pays ~105 ms to boot before it reaches main.
16+"""
17+import collections, subprocess, sys, time
18+
19+BENCHES = ["fib", "loop", "seqs"]
20+ROUNDS = 11
21+
22+
23+def timed(cmd):
24+ start = time.perf_counter()
25+ subprocess.run(cmd, capture_output=True)
26+ return (time.perf_counter() - start) * 1000
27+
28+
29+def main(argv):
30+ # {label: {bench: argv}} -- "hello" is the startup probe every label needs.
31+ suites = {"clonim": {b: [f"bench/bin/{b}"] for b in ["hello"] + BENCHES}}
32+ if "--with-jolt" in argv:
33+ suites["jolt"] = {b: [f"bench/bin/jolt-{b}"] for b in ["hello"] + BENCHES}
34+
35+ best = collections.defaultdict(lambda: float("inf"))
36+ for label, cmds in suites.items(): # warm the page cache first
37+ for cmd in cmds.values():
38+ subprocess.run(cmd, capture_output=True)
39+ for _ in range(ROUNDS):
40+ for label, cmds in suites.items():
41+ for bench, cmd in cmds.items():
42+ best[label, bench] = min(best[label, bench], timed(cmd))
43+
44+ width = max(len(l) for l in suites)
45+ print(f"{'':<{width}} {'startup':>9} " + " ".join(f"{b:>8}" for b in BENCHES))
46+ for label in suites:
47+ start = best[label, "hello"]
48+ row = " ".join(f"{best[label, b] - start:8.0f}" for b in BENCHES)
49+ print(f"{label:<{width}} {start:8.1f}ms {row}")
50+ print("\nstartup is total wall clock; the rest is compute, startup subtracted.")
51+
52+
53+if __name__ == "__main__":
54+ main(sys.argv[1:])
new file mode 100755
@@ -0,0 +1,54 @@
1+#!/usr/bin/env python3
2+"""Time clonim's benchmark binaries, optionally against jolt.
3+
4+Two things make these numbers trustworthy rather than merely small:
5+
6+ * best-of-N, not mean. A benchmark process contends with whatever else the
7+ machine is doing; contention only ever adds time, so the minimum is the
8+ cleanest estimate of the work itself.
9+ * round-robin, not grouped. Running every binary once per round, rather than
10+ one binary N times before moving on, means a slow stretch of machine time
11+ lands on all of them instead of whichever happened to be running.
12+
13+Compute is reported as total minus the `hello` time for the same toolchain,
14+which is how a 0.5 ms native start is kept from flattering clonim against a
15+runtime that pays ~105 ms to boot before it reaches main.
16+"""
17+import collections, subprocess, sys, time
18+
19+BENCHES = ["fib", "loop", "seqs"]
20+ROUNDS = 11
21+
22+
23+def timed(cmd):
24+ start = time.perf_counter()
25+ subprocess.run(cmd, capture_output=True)
26+ return (time.perf_counter() - start) * 1000
27+
28+
29+def main(argv):
30+ # {label: {bench: argv}} -- "hello" is the startup probe every label needs.
31+ suites = {"clonim": {b: [f"bench/bin/{b}"] for b in ["hello"] + BENCHES}}
32+ if "--with-jolt" in argv:
33+ suites["jolt"] = {b: [f"bench/bin/jolt-{b}"] for b in ["hello"] + BENCHES}
34+
35+ best = collections.defaultdict(lambda: float("inf"))
36+ for label, cmds in suites.items(): # warm the page cache first
37+ for cmd in cmds.values():
38+ subprocess.run(cmd, capture_output=True)
39+ for _ in range(ROUNDS):
40+ for label, cmds in suites.items():
41+ for bench, cmd in cmds.items():
42+ best[label, bench] = min(best[label, bench], timed(cmd))
43+
44+ width = max(len(l) for l in suites)
45+ print(f"{'':<{width}} {'startup':>9} " + " ".join(f"{b:>8}" for b in BENCHES))
46+ for label in suites:
47+ start = best[label, "hello"]
48+ row = " ".join(f"{best[label, b] - start:8.0f}" for b in BENCHES)
49+ print(f"{label:<{width}} {start:8.1f}ms {row}")
50+ print("\nstartup is total wall clock; the rest is compute, startup subtracted.")
51+
52+
53+if __name__ == "__main__":
54+ main(sys.argv[1:])
added bench/seqs.clj +3 -0
new file mode 100644
@@ -0,0 +1,3 @@
1+(defn work [n]
2+ (reduce + (map (fn [x] (* x x)) (filter even? (range n)))))
3+(println (work 1000000))
new file mode 100644
@@ -0,0 +1,3 @@
1+(defn work [n]
2+ (reduce + (map (fn [x] (* x x)) (filter even? (range n)))))
3+(println (work 1000000))
added examples/directcalls.clj +20 -0
new file mode 100644
@@ -0,0 +1,20 @@
1+(defn f [x] (* x 2))
2+(println (f 5))
3+(def f (fn [x] (+ x 100)))
4+(println (f 5)) ; rebinding must win: 105
5+
6+(defn g ([x] (g x 1)) ([x y] (+ x y)) ([x y & more] (reduce + (cons (+ x y) more))))
7+(println (g 3) (g 3 4) (g 1 2 3 4)) ; 4 7 10
8+
9+(defn h [x] (inc x))
10+(println (map h [1 2 3])) ; h as a value: (2 3 4)
11+
12+(defn shadow [x] x)
13+(println (let [shadow (fn [y] (* y 10))] (shadow 5))) ; local shadows: 50
14+
15+(defn fact [n] (if (< n 2) 1 (* n (fact (- n 1)))))
16+(println (fact 10)) ; 3628800
17+
18+(defn ev? [n] (if (= n 0) true (od? (- n 1))))
19+(defn od? [n] (if (= n 0) false (ev? (- n 1))))
20+(println (ev? 10) (od? 7)) ; true true
new file mode 100644
@@ -0,0 +1,20 @@
1+(defn f [x] (* x 2))
2+(println (f 5))
3+(def f (fn [x] (+ x 100)))
4+(println (f 5)) ; rebinding must win: 105
5+
6+(defn g ([x] (g x 1)) ([x y] (+ x y)) ([x y & more] (reduce + (cons (+ x y) more))))
7+(println (g 3) (g 3 4) (g 1 2 3 4)) ; 4 7 10
8+
9+(defn h [x] (inc x))
10+(println (map h [1 2 3])) ; h as a value: (2 3 4)
11+
12+(defn shadow [x] x)
13+(println (let [shadow (fn [y] (* y 10))] (shadow 5))) ; local shadows: 50
14+
15+(defn fact [n] (if (< n 2) 1 (* n (fact (- n 1)))))
16+(println (fact 10)) ; 3628800
17+
18+(defn ev? [n] (if (= n 0) true (od? (- n 1))))
19+(defn od? [n] (if (= n 0) false (ev? (- n 1))))
20+(println (ev? 10) (od? 7)) ; true true
added examples/evalorder.clj +38 -0
new file mode 100644
@@ -0,0 +1,38 @@
1+(def log (atom []))
2+(defn note [x] (do (reset! log (conj (deref log) x)) x))
3+
4+;; 1. argument evaluation order, left to right
5+(reset! log [])
6+(defn three [a b c] (str a b c))
7+(println (three (note 1) (note 2) (note 3)) (deref log))
8+
9+;; 2. side effects in non-tail positions of a body must still run
10+(reset! log [])
11+(defn body [] (do (note :a) (note :b) :done))
12+(println (body) (deref log))
13+
14+;; 3. mixed pure and statement-shaped arguments keep their order
15+(reset! log [])
16+(println (three (note 1) (if true (note 2) nil) (note 3)) (deref log))
17+
18+;; 4. recur arguments see the OLD bindings
19+(println (loop [i 0 acc 0] (if (< i 5) (recur (+ i 1) (+ acc i)) acc))) ; 10
20+(println (loop [a 1 b 2 n 0] (if (< n 3) (recur b a (+ n 1)) [a b]))) ; swapped 3x -> [2 1]
21+;; 5. when-let / if-let evaluate the test exactly once
22+(reset! log [])
23+(println (when-let [v (note 7)] v) (deref log))
24+(reset! log [])
25+(println (if-let [v (note 8)] v :no) (deref log))
26+
27+;; 6. and / or short-circuit
28+(reset! log [])
29+(println (and false (note :never)) (or :first (note :never2)) (deref log))
30+
31+;; 7. collection literal element order
32+(reset! log [])
33+(println [(note 1) (note 2)] (deref log))
34+(reset! log [])
35+(println (count {(note :k1) (note :v1)}) (deref log))
36+
37+;; 8. a let binding shadowing mid-body
38+(println (let [x 1 y (+ x 1) x (+ y 10)] [x y])) ; [12 2]
new file mode 100644
@@ -0,0 +1,38 @@
1+(def log (atom []))
2+(defn note [x] (do (reset! log (conj (deref log) x)) x))
3+
4+;; 1. argument evaluation order, left to right
5+(reset! log [])
6+(defn three [a b c] (str a b c))
7+(println (three (note 1) (note 2) (note 3)) (deref log))
8+
9+;; 2. side effects in non-tail positions of a body must still run
10+(reset! log [])
11+(defn body [] (do (note :a) (note :b) :done))
12+(println (body) (deref log))
13+
14+;; 3. mixed pure and statement-shaped arguments keep their order
15+(reset! log [])
16+(println (three (note 1) (if true (note 2) nil) (note 3)) (deref log))
17+
18+;; 4. recur arguments see the OLD bindings
19+(println (loop [i 0 acc 0] (if (< i 5) (recur (+ i 1) (+ acc i)) acc))) ; 10
20+(println (loop [a 1 b 2 n 0] (if (< n 3) (recur b a (+ n 1)) [a b]))) ; swapped 3x -> [2 1]
21+;; 5. when-let / if-let evaluate the test exactly once
22+(reset! log [])
23+(println (when-let [v (note 7)] v) (deref log))
24+(reset! log [])
25+(println (if-let [v (note 8)] v :no) (deref log))
26+
27+;; 6. and / or short-circuit
28+(reset! log [])
29+(println (and false (note :never)) (or :first (note :never2)) (deref log))
30+
31+;; 7. collection literal element order
32+(reset! log [])
33+(println [(note 1) (note 2)] (deref log))
34+(reset! log [])
35+(println (count {(note :k1) (note :v1)}) (deref log))
36+
37+;; 8. a let binding shadowing mid-body
38+(println (let [x 1 y (+ x 1) x (+ y 10)] [x y])) ; [12 2]
added examples/forms.clj +10 -0
new file mode 100644
@@ -0,0 +1,10 @@
1+(def n (atom 0))
2+(dotimes [i (+ 2 3)] (reset! n (+ (deref n) i)))
3+(println "dotimes" (deref n)) ; 0+1+2+3+4 = 10
4+(doseq [x (map inc [1 2 3])] (reset! n (+ (deref n) x)))
5+(println "doseq" (deref n)) ; 10 + 2+3+4 = 19
6+(println (try (/ 1 0) (catch Exception e (str "caught: " e)) (finally (reset! n 99))))
7+(println "finally ran:" (deref n))
8+(println (-> 5 (+ 1) (* 2)) (->> [1 2 3] (map inc) (reduce +))) ; 12 9
9+(println (cond (> 1 2) :a (< 1 2) :b :else :c)) ; :b
10+(println (str "nested " (+ 1 (* 2 (- 10 (+ 3 4)))))) ; 1+2*3 = 7
new file mode 100644
@@ -0,0 +1,10 @@
1+(def n (atom 0))
2+(dotimes [i (+ 2 3)] (reset! n (+ (deref n) i)))
3+(println "dotimes" (deref n)) ; 0+1+2+3+4 = 10
4+(doseq [x (map inc [1 2 3])] (reset! n (+ (deref n) x)))
5+(println "doseq" (deref n)) ; 10 + 2+3+4 = 19
6+(println (try (/ 1 0) (catch Exception e (str "caught: " e)) (finally (reset! n 99))))
7+(println "finally ran:" (deref n))
8+(println (-> 5 (+ 1) (* 2)) (->> [1 2 3] (map inc) (reduce +))) ; 12 9
9+(println (cond (> 1 2) :a (< 1 2) :b :else :c)) ; :b
10+(println (str "nested " (+ 1 (* 2 (- 10 (+ 3 4)))))) ; 1+2*3 = 7
added examples/intrinsics.clj +11 -0
new file mode 100644
@@ -0,0 +1,11 @@
1+(println (+ 1 2) (- 5 3) (* 4 5) (inc 7) (dec 7)) ; 3 2 20 8 6
2+(println (+ 1.5 2) (* 2 0.5) (- 1 0.25) (inc 1.5)) ; 3.5 1.0 0.75 2.5
3+(println (< 1 2) (> 1 2) (<= 2 2) (>= 1 2)) ; true false true false
4+(println (< 1.5 2) (>= 2.0 2)) ; true true
5+(println (= 1 1) (= 1 2) (= "a" "a") (= [1 2] [1 2])) ; true false true true
6+(println (= 1 1.0) (not= 1 2) (not= :a :a)) ; true true false
7+(println (+ 1 2 3) (< 1 2 3) (= 1 1 1)) ; non-binary arities: 6 true true
8+(println (let [+ (fn [a b] (* a b))] (+ 3 4))) ; local shadow: 12
9+(println (reduce + [1 2 3 4])) ; + as a value: 10
10+(def + (fn [a b] 999))
11+(println (+ 1 2)) ; rebound: 999
new file mode 100644
@@ -0,0 +1,11 @@
1+(println (+ 1 2) (- 5 3) (* 4 5) (inc 7) (dec 7)) ; 3 2 20 8 6
2+(println (+ 1.5 2) (* 2 0.5) (- 1 0.25) (inc 1.5)) ; 3.5 1.0 0.75 2.5
3+(println (< 1 2) (> 1 2) (<= 2 2) (>= 1 2)) ; true false true false
4+(println (< 1.5 2) (>= 2.0 2)) ; true true
5+(println (= 1 1) (= 1 2) (= "a" "a") (= [1 2] [1 2])) ; true false true true
6+(println (= 1 1.0) (not= 1 2) (not= :a :a)) ; true true false
7+(println (+ 1 2 3) (< 1 2 3) (= 1 1 1)) ; non-binary arities: 6 true true
8+(println (let [+ (fn [a b] (* a b))] (+ 3 4))) ; local shadow: 12
9+(println (reduce + [1 2 3 4])) ; + as a value: 10
10+(def + (fn [a b] 999))
11+(println (+ 1 2)) ; rebound: 999
added examples/rebinding.clj +13 -0
new file mode 100644
@@ -0,0 +1,13 @@
1+;; a fn redefined later: every call site sees the definition in force
2+(defn f [x] (+ x 1))
3+(println (f 10)) ; 11
4+(def f (fn [x] (* x 100)))
5+(println (f 10)) ; 1000
6+
7+;; redefining a primitive defeats specialisation for the whole program, so the
8+;; loop below must observe the new + even though it was compiled earlier
9+(defn sum [n] (loop [i 0 acc 0] (if (< i n) (recur (inc i) (+ acc i)) acc)))
10+(println (sum 5)) ; 0+1+2+3+4 = 10
11+(def + (fn [a b] (* a b)))
12+(println (sum 5)) ; + is now *, so acc stays 0
13+(println (+ 3 4)) ; 12
new file mode 100644
@@ -0,0 +1,13 @@
1+;; a fn redefined later: every call site sees the definition in force
2+(defn f [x] (+ x 1))
3+(println (f 10)) ; 11
4+(def f (fn [x] (* x 100)))
5+(println (f 10)) ; 1000
6+
7+;; redefining a primitive defeats specialisation for the whole program, so the
8+;; loop below must observe the new + even though it was compiled earlier
9+(defn sum [n] (loop [i 0 acc 0] (if (< i n) (recur (inc i) (+ acc i)) acc)))
10+(println (sum 5)) ; 0+1+2+3+4 = 10
11+(def + (fn [a b] (* a b)))
12+(println (sum 5)) ; + is now *, so acc stays 0
13+(println (+ 3 4)) ; 12
added examples/seqlib.clj +17 -0
new file mode 100644
@@ -0,0 +1,17 @@
1+;; range: arities, steps, empty and negative
2+(println (range 5) (range 2 5) (range 0 10 3) (range 5 0 -2))
3+(println (range 0) (range 5 5) (range 5 0) (range 0 5 -1))
4+;; count / empty? across kinds
5+(println (count [1 2 3]) (count '(1 2)) (count "abcd") (count {:a 1 :b 2}) (count nil))
6+(println (empty? []) (empty? [1]) (empty? "") (empty? nil))
7+;; non-fn callables must still work as the function argument
8+(println (map :a [{:a 1} {:a 2}])) ; (1 2)
9+(println (map {1 :one 2 :two} [1 2])) ; (:one :two)
10+;; reduce: empty, single, with an explicit init
11+(println (reduce + []) (reduce + [7]) (reduce + 100 [1 2 3]) (reduce + [1 2 3]))
12+;; map/filter over other collection kinds
13+(println (map inc #{1 2}) (count (filter even? (range 10))) (map inc "") )
14+;; laziness is not claimed: these are eager, so side effects happen once
15+(def n (atom 0))
16+(def r (map (fn [x] (reset! n (+ (deref n) 1)) x) [1 2 3]))
17+(println (count r) (deref n) (count r) (deref n)) ; 3 3 3 3
new file mode 100644
@@ -0,0 +1,17 @@
1+;; range: arities, steps, empty and negative
2+(println (range 5) (range 2 5) (range 0 10 3) (range 5 0 -2))
3+(println (range 0) (range 5 5) (range 5 0) (range 0 5 -1))
4+;; count / empty? across kinds
5+(println (count [1 2 3]) (count '(1 2)) (count "abcd") (count {:a 1 :b 2}) (count nil))
6+(println (empty? []) (empty? [1]) (empty? "") (empty? nil))
7+;; non-fn callables must still work as the function argument
8+(println (map :a [{:a 1} {:a 2}])) ; (1 2)
9+(println (map {1 :one 2 :two} [1 2])) ; (:one :two)
10+;; reduce: empty, single, with an explicit init
11+(println (reduce + []) (reduce + [7]) (reduce + 100 [1 2 3]) (reduce + [1 2 3]))
12+;; map/filter over other collection kinds
13+(println (map inc #{1 2}) (count (filter even? (range 10))) (map inc "") )
14+;; laziness is not claimed: these are eager, so side effects happen once
15+(def n (atom 0))
16+(def r (map (fn [x] (reset! n (+ (deref n) 1)) x) [1 2 3]))
17+(println (count r) (deref n) (count r) (deref n)) ; 3 3 3 3
added examples/typespec.clj +35 -0
new file mode 100644
@@ -0,0 +1,35 @@
1+;; loop vars that must NOT specialise
2+(println (loop [x 0.0 n 0] (if (< n 3) (recur (+ x 0.5) (+ n 1)) x))) ; 1.5
3+(println (loop [s "" n 0] (if (< n 3) (recur (str s "a") (+ n 1)) s))) ; "aaa"
4+(println (loop [v [] n 0] (if (< n 3) (recur (conj v n) (+ n 1)) v))) ; [0 1 2]
5+;; a var that starts int but recurs with a float must demote
6+(println (loop [x 0 n 0] (if (< n 3) (recur (+ x 0.5) (+ n 1)) x))) ; 1.5
7+;; mixed: one int slot, one not
8+(println (loop [i 0 acc []] (if (< i 3) (recur (+ i 1) (conj acc i)) acc))) ; [0 1 2]
9+
10+;; int fns called with non-int arguments must use the generic path
11+(defn twice [x] (+ x x))
12+(println (twice 21) (twice 1.5)) ; 42 3.0
13+(defn joiner [a b] (str a b))
14+(println (joiner 1 2) (joiner "a" "b")) ; 12 ab
15+;; a fn used as a value still works
16+(println (map twice [1 2 3]) (reduce + (map twice [1 2]))) ; (2 4 6) 6
17+
18+;; shadowing a primitive inside a fn body defeats specialisation
19+(defn shadowed [x] (let [+ (fn [a b] (* a b))] (+ x x)))
20+(println (shadowed 5)) ; 25
21+
22+;; quot / rem / inc / dec, including negatives
23+(println (quot 7 2) (quot -7 2) (rem 7 2) (rem -7 2) (inc 5) (dec 5)) ; 3 -3 1 -1 6 4
24+;; comparison chain of non-numbers must not become an int compare
25+(println (= "a" "a") (= :k :k) (= [1] [1]) (not= 1 2)) ; true true true true
26+;; division by zero still reports, rather than trapping
27+(println (try (quot 1 0) (catch Exception e "caught")))
28+;; a fn whose recur makes a parameter non-integral must not specialise
29+(defn k [x n] (if (< n 3) (recur (str x "a") (+ n 1)) x))
30+(println (k "" 0)) ; aaa
31+;; recur that keeps every parameter integral still specialises, and is correct
32+(defn countdown [n acc] (if (< n 1) acc (recur (- n 1) (+ acc n))))
33+(println (countdown 100 0)) ; 5050
34+;; deep tail recursion must stay in constant space
35+(println (countdown 3000000 0)) ; 4500001500000
new file mode 100644
@@ -0,0 +1,35 @@
1+;; loop vars that must NOT specialise
2+(println (loop [x 0.0 n 0] (if (< n 3) (recur (+ x 0.5) (+ n 1)) x))) ; 1.5
3+(println (loop [s "" n 0] (if (< n 3) (recur (str s "a") (+ n 1)) s))) ; "aaa"
4+(println (loop [v [] n 0] (if (< n 3) (recur (conj v n) (+ n 1)) v))) ; [0 1 2]
5+;; a var that starts int but recurs with a float must demote
6+(println (loop [x 0 n 0] (if (< n 3) (recur (+ x 0.5) (+ n 1)) x))) ; 1.5
7+;; mixed: one int slot, one not
8+(println (loop [i 0 acc []] (if (< i 3) (recur (+ i 1) (conj acc i)) acc))) ; [0 1 2]
9+
10+;; int fns called with non-int arguments must use the generic path
11+(defn twice [x] (+ x x))
12+(println (twice 21) (twice 1.5)) ; 42 3.0
13+(defn joiner [a b] (str a b))
14+(println (joiner 1 2) (joiner "a" "b")) ; 12 ab
15+;; a fn used as a value still works
16+(println (map twice [1 2 3]) (reduce + (map twice [1 2]))) ; (2 4 6) 6
17+
18+;; shadowing a primitive inside a fn body defeats specialisation
19+(defn shadowed [x] (let [+ (fn [a b] (* a b))] (+ x x)))
20+(println (shadowed 5)) ; 25
21+
22+;; quot / rem / inc / dec, including negatives
23+(println (quot 7 2) (quot -7 2) (rem 7 2) (rem -7 2) (inc 5) (dec 5)) ; 3 -3 1 -1 6 4
24+;; comparison chain of non-numbers must not become an int compare
25+(println (= "a" "a") (= :k :k) (= [1] [1]) (not= 1 2)) ; true true true true
26+;; division by zero still reports, rather than trapping
27+(println (try (quot 1 0) (catch Exception e "caught")))
28+;; a fn whose recur makes a parameter non-integral must not specialise
29+(defn k [x n] (if (< n 3) (recur (str x "a") (+ n 1)) x))
30+(println (k "" 0)) ; aaa
31+;; recur that keeps every parameter integral still specialises, and is correct
32+(defn countdown [n acc] (if (< n 1) acc (recur (- n 1) (+ acc n))))
33+(println (countdown 100 0)) ; 5050
34+;; deep tail recursion must stay in constant space
35+(println (countdown 3000000 0)) ; 4500001500000
modified justfile +44 -1
@@ -44,4 +44,47 @@ accept: build
4444
4545 # Remove build output
4646 clean:
47- rm -rf bin nimcache
47+ rm -rf bin nimcache bench/bin
48+
49+# ---------------------------------------------------------------- benchmarks
50+#
51+# `just bench` above stays as it was: the two feature benchmarks, run through
52+# the compiler with timing printed by the programs themselves. What follows
53+# times whole processes instead, which is the only way to see startup, and can
54+# put another Clojure toolchain beside clonim.
55+
56+# `just measure jolt` also builds and times the same programs under jolt, if it
57+# is installed. Both toolchains get a `hello` binary so startup can be
58+# subtracted; see bench/measure.py for why the numbers are best-of-N
59+# round-robin rather than averaged.
60+# Time bench/*.clj as native binaries, optionally against jolt.
61+measure mode="solo": release
62+ #!/usr/bin/env bash
63+ set -euo pipefail
64+ mkdir -p bench/bin
65+ for f in bench/*.clj; do
66+ name=$(basename "$f" .clj)
67+ ./{{bin}} build "$f" -o "bench/bin/$name" >/dev/null
68+ done
69+ if [ "{{mode}}" != "jolt" ]; then
70+ exec python3 bench/measure.py
71+ fi
72+ if ! command -v jolt >/dev/null; then
73+ echo "jolt is not installed; running clonim only" >&2
74+ exec python3 bench/measure.py
75+ fi
76+ # jolt builds a namespace, not a file, so mirror each bench into a deps.edn
77+ # project under a throwaway directory, wrapping the last form in a -main.
78+ work=$(mktemp -d)
79+ trap 'rm -rf "$work"' EXIT
80+ mkdir -p "$work/src"
81+ echo '{:paths ["src"]}' > "$work/deps.edn"
82+ for f in bench/*.clj; do
83+ name=$(basename "$f" .clj)
84+ { echo "(ns b$name)"
85+ sed 's/^(println \(.*\))$/(defn -main [\& _] (println \1))/' "$f"
86+ } > "$work/src/b$name.clj"
87+ (cd "$work" && jolt build -m "b$name" -o "b$name" --opt >/dev/null)
88+ cp "$work/b$name" "bench/bin/jolt-$name"
89+ done
90+ python3 bench/measure.py --with-jolt
@@ -44,4 +44,47 @@ accept: build
44 44
45 # Remove build output45 # Remove build output
46 clean:46 clean:
47- rm -rf bin nimcache47+ rm -rf bin nimcache bench/bin
48+
49+# ---------------------------------------------------------------- benchmarks
50+#
51+# `just bench` above stays as it was: the two feature benchmarks, run through
52+# the compiler with timing printed by the programs themselves. What follows
53+# times whole processes instead, which is the only way to see startup, and can
54+# put another Clojure toolchain beside clonim.
55+
56+# `just measure jolt` also builds and times the same programs under jolt, if it
57+# is installed. Both toolchains get a `hello` binary so startup can be
58+# subtracted; see bench/measure.py for why the numbers are best-of-N
59+# round-robin rather than averaged.
60+# Time bench/*.clj as native binaries, optionally against jolt.
61+measure mode="solo": release
62+ #!/usr/bin/env bash
63+ set -euo pipefail
64+ mkdir -p bench/bin
65+ for f in bench/*.clj; do
66+ name=$(basename "$f" .clj)
67+ ./{{bin}} build "$f" -o "bench/bin/$name" >/dev/null
68+ done
69+ if [ "{{mode}}" != "jolt" ]; then
70+ exec python3 bench/measure.py
71+ fi
72+ if ! command -v jolt >/dev/null; then
73+ echo "jolt is not installed; running clonim only" >&2
74+ exec python3 bench/measure.py
75+ fi
76+ # jolt builds a namespace, not a file, so mirror each bench into a deps.edn
77+ # project under a throwaway directory, wrapping the last form in a -main.
78+ work=$(mktemp -d)
79+ trap 'rm -rf "$work"' EXIT
80+ mkdir -p "$work/src"
81+ echo '{:paths ["src"]}' > "$work/deps.edn"
82+ for f in bench/*.clj; do
83+ name=$(basename "$f" .clj)
84+ { echo "(ns b$name)"
85+ sed 's/^(println \(.*\))$/(defn -main [\& _] (println \1))/' "$f"
86+ } > "$work/src/b$name.clj"
87+ (cd "$work" && jolt build -m "b$name" -o "b$name" --opt >/dev/null)
88+ cp "$work/b$name" "bench/bin/jolt-$name"
89+ done
90+ python3 bench/measure.py --with-jolt
modified src/compiler.nim +558 -60
@@ -7,21 +7,37 @@ import std/[tables, strutils, sets]
77 import runtime, reader
88
99 type
10+ ## A fn whose arity is known at the call site, so it can be reached as a
11+ ## plain Nim proc instead of through a seq of boxed args.
12+ Direct = object
13+ prc: string # nim proc ident taking positional Values
14+ cell: string # var cell to guard on ("" = no guard)
15+ fnVal: string # nim ident of the fn Value to compare
16+
1017 Env = ref object
1118 parent: Env
1219 locals: Table[string, string] # clojure name -> nim identifier
20+ ints: HashSet[string] # of those, the ones held as raw int64
21+ directs: Table[string, Direct] # "name/arity" -> positional entry point
22+ intFns: Table[string, string] # "name/arity" -> int-specialised proc
23+ intFnBoxes: HashSet[string] # of those, the ones returning Value
1324
1425 Ctx = ref object
1526 body: seq[string] # emitted lines
1627 indent: int
1728 counter: int
1829 recurStack: seq[seq[string]] # nim idents of the enclosing recur target
30+ recurInts: seq[seq[bool]] # which of those are raw int64
1931 defined: HashSet[string] # names def'd so far (for nicer errors)
2032 prelude: seq[string] # hoisted var-cell resolutions
2133 cells: Table[string, string] # clojure var name -> nim cell ident
34+ cores: Table[string, string] # var name -> nim ident holding its core fn
35+ defCounts: CountTable[string] # how many def forms target each name
2236
2337 proc newEnv(parent: Env = nil): Env =
24- Env(parent: parent, locals: initTable[string, string]())
38+ Env(parent: parent, locals: initTable[string, string](),
39+ ints: initHashSet[string](), directs: initTable[string, Direct](),
40+ intFns: initTable[string, string](), intFnBoxes: initHashSet[string]())
2541
2642 proc lookup(env: Env, name: string): string =
2743 var e = env
@@ -30,6 +46,45 @@ proc lookup(env: Env, name: string): string =
3046 e = e.parent
3147 ""
3248
49+proc isIntLocal(env: Env, name: string): bool =
50+ ## True when the name's Nim binding is an int64 rather than a Value.
51+ var e = env
52+ while e != nil:
53+ if e.locals.hasKey(name): return e.ints.contains(name)
54+ e = e.parent
55+ false
56+
57+proc lookupIntFn(env: Env, name: string, arity: int): (string, bool) =
58+ ## The int-specialised entry point for a fn, and whether it returns a Value
59+ ## (true) or a raw int64 (false).
60+ let key = name & "/" & $arity
61+ var e = env
62+ while e != nil:
63+ if e.intFns.hasKey(key): return (e.intFns[key], e.intFnBoxes.contains(key))
64+ if e.locals.hasKey(name): return ("", false)
65+ e = e.parent
66+ ("", false)
67+
68+proc lookupDirect(env: Env, name: string, arity: int): Direct =
69+ ## A local binding of the same name shadows the direct entry point.
70+ let key = name & "/" & $arity
71+ var e = env
72+ while e != nil:
73+ if e.directs.hasKey(key): return e.directs[key]
74+ if e.locals.hasKey(name): return Direct()
75+ e = e.parent
76+ Direct()
77+
78+proc primStable(c: Ctx, name: string): bool =
79+ ## A core builtin still holds its registerCore value everywhere: no def in
80+ ## this program targets the name at all.
81+ c.defCounts[name] == 0
82+
83+proc fnStable(c: Ctx, name: string): bool =
84+ ## A user fn is reached by exactly one definition, so the one a call site was
85+ ## compiled against is the only one it can ever see.
86+ c.defCounts[name] <= 1
87+
3388 proc line(c: Ctx, s: string) =
3489 c.body.add repeat(" ", c.indent) & s
3590
@@ -70,6 +125,36 @@ proc cellFor(c: Ctx, name: string): string =
70125 c.cells[name] = result
71126 c.prelude.add " let " & result & " = varCell(" & nimStr(name) & ")"
72127
128+## Builtins with a cheap inline form. A call site at the matching arity emits
129+## the inline proc directly, guarded on the var still holding the core fn.
130+const intrinsics = {
131+ "+/2": "add2", "-/2": "sub2", "*/2": "mul2",
132+ "</2": "lt2", ">/2": "gt2", "<=/2": "le2", ">=/2": "ge2",
133+ "=/2": "eq2", "not=/2": "ne2", "inc/1": "inc1", "dec/1": "dec1",
134+}.toTable
135+
136+## Heads that compile to statements, never to a single expression.
137+const specialHeads = ["quote", "if", "do", "let", "let*", "loop", "loop*",
138+ "recur", "fn", "fn*", "def", "defn", "defn-", "defmacro", "and", "or",
139+ "when", "when-not", "if-not", "cond", "when-let", "if-let", "->", "->>",
140+ "doseq", "dotimes", "try", "comment", "ns", "require", "in-ns", "use",
141+ "import", "set!", "declare"].toHashSet
142+
143+const intOps = {"+": "+", "-": "-", "*": "*"}.toTable
144+const intCalls = {"quot": "idiv", "rem": "irem"}.toTable
145+const cmpOps = {"<": "<", ">": ">", "<=": "<=", ">=": ">=", "=": "==",
146+ "not=": "!="}.toTable
147+
148+proc coreFor(c: Ctx, name: string): string =
149+ ## The value a core builtin had at program start, captured once, so call
150+ ## sites can tell whether the var still holds it.
151+ if c.cores.hasKey(name): return c.cores[name]
152+ let cell = c.cellFor(name)
153+ inc c.counter
154+ result = "k_" & $c.counter
155+ c.cores[name] = result
156+ c.prelude.add " let " & result & " = cellGet(" & cell & ")"
157+
73158 proc isSym(v: Value, name: string): bool =
74159 not v.isNil and v.kind == kSymbol and v.s == name
75160
@@ -110,18 +195,53 @@ proc emptySeqFix(s: string, elemType: string): string =
110195
111196 # ------------------------------------------------------------- code gen
112197 proc genInto(f: Value, dst: string, env: Env, c: Ctx)
198+proc tryExpr(f: Value, env: Env, c: Ctx): string
199+proc intExpr(f: Value, env: Env, c: Ctx): string
200+proc boolExpr(f: Value, env: Env, c: Ctx): string
113201
114202 proc genExpr(f: Value, env: Env, c: Ctx): string =
203+ ## A Nim expression denoting the form's value. Forms that compile to a single
204+ ## expression are returned as-is; the rest go through a temporary slot.
205+ result = tryExpr(f, env, c)
206+ if result.len > 0: return
115207 result = c.gensym("t")
116208 c.line("var " & result & ": Value = NilV")
117209 genInto(f, result, env, c)
118210
211+proc genExprTemp(f: Value, env: Env, c: Ctx): string =
212+ ## Like genExpr, but always materialises the value into a fresh local. Used
213+ ## where the value is read more than once, or must be computed before a
214+ ## later statement can overwrite what it reads.
215+ let e = tryExpr(f, env, c)
216+ if e.len > 0:
217+ result = c.gensym("t")
218+ c.line("let " & result & ": Value = " & e)
219+ return
220+ result = c.gensym("t")
221+ c.line("var " & result & ": Value = NilV")
222+ genInto(f, result, env, c)
223+
224+proc genCond(f: Value, env: Env, c: Ctx): string =
225+ ## A Nim bool for a test position. A comparison between provable integers
226+ ## becomes a machine compare; anything else falls back to truthy() on a Value.
227+ result = boolExpr(f, env, c)
228+ if result.len > 0: return
229+ result = "truthy(" & genExpr(f, env, c) & ")"
230+
231+proc genStmt(f: Value, env: Env, c: Ctx) =
232+ ## A form evaluated only for its effects.
233+ let e = tryExpr(f, env, c)
234+ if e.len > 0:
235+ c.line("discard " & e)
236+ return
237+ discard genExpr(f, env, c)
238+
119239 proc genBody(forms: seq[Value], dst: string, env: Env, c: Ctx) =
120240 if forms.len == 0:
121241 c.line(dst & " = NilV")
122242 return
123243 for i in 0 ..< forms.len - 1:
124- discard genExpr(forms[i], env, c)
244+ genStmt(forms[i], env, c)
125245 genInto(forms[^1], dst, env, c)
126246
127247 type
@@ -143,41 +263,170 @@ proc parseParams(v: Value): FnClause =
143263 result.params.add symName(p)
144264 inc i
145265
146-proc genFn(name: string, clauses: seq[FnClause], selfIdent: string, env: Env, c: Ctx, dst: string) =
266+proc genClauseBody(cl: FnClause, name, selfIdent: string, env: Env, c: Ctx,
267+ bindTo: proc (i: int, p: string): string, res: string,
268+ selfDirect = "", intParams = false) =
269+ ## Shared between the positional proc and the generic dispatcher: bind the
270+ ## params to mutable locals (recur assigns them), then run the body.
271+ let fenv = newEnv(env)
272+ if selfIdent.len > 0 and name.len > 0:
273+ fenv.locals[name] = selfIdent
274+ # Registered alongside the self local, and checked first, so a self-call at
275+ # the matching arity reaches the positional proc rather than the fn Value.
276+ if selfDirect.len > 0:
277+ fenv.directs[name & "/" & $cl.params.len] = Direct(prc: selfDirect)
278+ var recurIdents: seq[string] = @[]
279+ for i, p in cl.params:
280+ let id = c.gensym("p" & mangle(p))
281+ c.line("var " & id & (if intParams: ": int64 = " else: ": Value = ") &
282+ bindTo(i, p))
283+ fenv.locals[p] = id
284+ if intParams: fenv.ints.incl p
285+ recurIdents.add id
286+ if cl.restParam.len > 0:
287+ let id = c.gensym("p" & mangle(cl.restParam))
288+ c.line("var " & id & ": Value = " & bindTo(-1, cl.restParam))
289+ fenv.locals[cl.restParam] = id
290+ c.line("var " & res & ": Value = NilV")
291+ c.line("while true:")
292+ c.push
293+ c.recurStack.add recurIdents
294+ var recurIsInt = newSeq[bool](recurIdents.len)
295+ if intParams:
296+ for j in 0 ..< recurIsInt.len: recurIsInt[j] = true
297+ c.recurInts.add recurIsInt
298+ genBody(cl.body, res, fenv, c)
299+ discard c.recurStack.pop
300+ discard c.recurInts.pop
301+ c.line("break")
302+ c.pop
303+
304+proc collectRecurs(forms: seq[Value], into: var seq[seq[Value]]) =
305+ ## The recur forms belonging to the innermost enclosing loop. Nested fn and
306+ ## loop forms establish their own recur target, so their bodies are skipped.
307+ for f in forms:
308+ if f.isNil or f.kind != kList or f.items.len == 0: continue
309+ let head = f.items[0]
310+ if head.kind == kSymbol:
311+ if head.s == "recur":
312+ into.add f.items[1 .. ^1]
313+ continue
314+ if head.s in ["loop", "loop*", "fn", "fn*", "defn", "defn-"]: continue
315+ collectRecurs(f.items, into)
316+
317+proc usesIntPrim(c: Ctx, forms: seq[Value]): bool =
318+ ## Whether an int-specialised twin could differ from the generic proc at all.
319+ ## Emitting one for a fn that never does arithmetic just doubles the work Nim
320+ ## has to do; this gate is an optimisation only, never a semantic decision.
321+ for f in forms:
322+ if f.isNil or f.kind != kList or f.items.len == 0: continue
323+ let head = f.items[0]
324+ if head.kind == kSymbol and c.primStable(head.s) and
325+ (intOps.hasKey(head.s) or intCalls.hasKey(head.s) or
326+ cmpOps.hasKey(head.s) or head.s == "inc" or head.s == "dec"):
327+ return true
328+ if usesIntPrim(c, f.items): return true
329+ false
330+
331+proc genFn(name: string, clauses: seq[FnClause], selfIdent: string, env: Env,
332+ c: Ctx, dst: string, directEnv: Env = nil, cell = "") =
333+ ## Each fixed-arity clause gets a real Nim proc taking its params
334+ ## positionally; the mkFn wrapper is just an arity dispatcher onto those, and
335+ ## call sites that know the arity skip the wrapper entirely.
336+ var directProcs: seq[string] = @[] # parallel to clauses, "" for variadic
337+ for cl in clauses:
338+ if cl.restParam.len > 0:
339+ directProcs.add ""
340+ continue
341+ let prc = c.gensym("uf" & mangle(if name.len > 0: name else: "fn"))
342+ var params: seq[string] = @[]
343+ for i in 0 ..< cl.params.len: params.add "a" & $i & ": Value"
344+ c.line("proc " & prc & "(" & params.join(", ") & "): Value =")
345+ c.push
346+ let res = c.gensym("res")
347+ # selfDirect makes a self-call at this arity a plain recursive Nim call.
348+ genClauseBody(cl, name, selfIdent, env, c,
349+ proc (i: int, p: string): string = "a" & $i, res, prc)
350+ c.line("return " & res)
351+ c.pop
352+ directProcs.add prc
353+ if directEnv != nil:
354+ directEnv.directs[name & "/" & $cl.params.len] =
355+ Direct(prc: prc, cell: cell, fnVal: dst)
356+
357+ # An int-specialised twin, so a caller with integer arguments never boxes
358+ # them. Emitted beside the generic proc rather than replacing it: callers
359+ # that cannot prove their arguments are integers still need the Value one.
360+ if name.len > 0 and cl.params.len > 0 and c.usesIntPrim(cl.body):
361+ let key = name & "/" & $cl.params.len
362+ let iprc = c.gensym("ufi" & mangle(name))
363+ # Does the body yield an integer? Probe with the parameters typed and the
364+ # fn optimistically assumed to return one, so self-recursion types too.
365+ # A recur inside the clause reassigns the parameters, so every recur
366+ # value has to stay integral too. Anything less and the twin is dropped
367+ # rather than emitted with a mix of int64 and Value parameters.
368+ let probe = newEnv(env)
369+ for i, p in cl.params:
370+ probe.locals[p] = "a" & $i
371+ probe.ints.incl p
372+ probe.intFns[key] = iprc
373+ var recurSafe = true
374+ var myRecurs: seq[seq[Value]] = @[]
375+ collectRecurs(cl.body, myRecurs)
376+ for r in myRecurs:
377+ if r.len != cl.params.len: recurSafe = false; break
378+ for a in r:
379+ if intExpr(a, probe, c).len == 0: recurSafe = false; break
380+ if not recurSafe: break
381+ var boxes = true
382+ if cl.body.len == 1 and intExpr(cl.body[0], probe, c).len > 0:
383+ boxes = false
384+ if recurSafe:
385+ var iparams: seq[string] = @[]
386+ for i in 0 ..< cl.params.len: iparams.add "a" & $i & ": int64"
387+ c.line("proc " & iprc & "(" & iparams.join(", ") & "): " &
388+ (if boxes: "Value" else: "int64") & " =")
389+ c.push
390+ let ienv = newEnv(env)
391+ ienv.intFns[key] = iprc
392+ if boxes: ienv.intFnBoxes.incl key
393+ if boxes:
394+ let ires = c.gensym("res")
395+ genClauseBody(cl, name, selfIdent, ienv, c,
396+ proc (i: int, p: string): string = "a" & $i, ires, "", true)
397+ c.line("return " & ires)
398+ else:
399+ for i, p in cl.params:
400+ ienv.locals[p] = "a" & $i
401+ ienv.ints.incl p
402+ c.line("return " & intExpr(cl.body[0], ienv, c))
403+ c.pop
404+ if directEnv != nil and c.fnStable(name):
405+ directEnv.intFns[key] = iprc
406+ if boxes: directEnv.intFnBoxes.incl key
407+
147408 let argsIdent = c.gensym("args")
148409 c.line(dst & " = mkFn(" & nimStr(name) & ", proc (" & argsIdent & ": seq[Value]): Value =")
149410 c.push
150411 var first = true
151- for cl in clauses:
412+ for ci, cl in clauses:
152413 let cond =
153414 if cl.restParam.len > 0: argsIdent & ".len >= " & $cl.params.len
154415 else: argsIdent & ".len == " & $cl.params.len
155416 c.line((if first: "if " else: "elif ") & cond & ":")
156417 first = false
157418 c.push
158- let fenv = newEnv(env)
159- if selfIdent.len > 0 and name.len > 0:
160- fenv.locals[name] = selfIdent
161- var recurIdents: seq[string] = @[]
162- for i, p in cl.params:
163- let id = c.gensym("p" & mangle(p))
164- c.line("var " & id & ": Value = argAt(" & argsIdent & ", " & $i & ")")
165- fenv.locals[p] = id
166- recurIdents.add id
167- if cl.restParam.len > 0:
168- let id = c.gensym("p" & mangle(cl.restParam))
169- c.line("var " & id & ": Value = restArgs(" & argsIdent & ", " & $cl.params.len & ")")
170- fenv.locals[cl.restParam] = id
171- let res = c.gensym("res")
172- c.line("var " & res & ": Value = NilV")
173- c.line("while true:")
174- c.push
175- c.recurStack.add recurIdents
176- genBody(cl.body, res, fenv, c)
177- discard c.recurStack.pop
178- c.line("break")
179- c.pop
180- c.line("return " & res)
419+ if directProcs[ci].len > 0:
420+ var fwd: seq[string] = @[]
421+ for i in 0 ..< cl.params.len: fwd.add "argAt(" & argsIdent & ", " & $i & ")"
422+ c.line("return " & directProcs[ci] & "(" & fwd.join(", ") & ")")
423+ else:
424+ let res = c.gensym("res")
425+ genClauseBody(cl, name, selfIdent, env, c,
426+ proc (i: int, p: string): string =
427+ if i < 0: "restArgs(" & argsIdent & ", " & $cl.params.len & ")"
428+ else: "argAt(" & argsIdent & ", " & $i & ")", res)
429+ c.line("return " & res)
181430 c.pop
182431 c.line("else:")
183432 c.push
@@ -187,7 +436,8 @@ proc genFn(name: string, clauses: seq[FnClause], selfIdent: string, env: Env, c:
187436 c.pop
188437 c.line(")")
189438
190-proc genFnForm(args: seq[Value], env: Env, c: Ctx, dst: string, defName: string) =
439+proc genFnForm(args: seq[Value], env: Env, c: Ctx, dst: string, defName: string,
440+ directEnv: Env = nil, cell = "") =
191441 ## (fn name? [params] body...) or (fn name? ([params] body...) ...)
192442 var i = 0
193443 var name = defName
@@ -212,7 +462,7 @@ proc genFnForm(args: seq[Value], env: Env, c: Ctx, dst: string, defName: string)
212462 # bind the fn to a local so it can recur by name
213463 selfIdent = c.gensym("self" & mangle(name))
214464 c.line("var " & selfIdent & ": Value = NilV")
215- genFn(name, clauses, selfIdent, env, c, selfIdent)
465+ genFn(name, clauses, selfIdent, env, c, selfIdent, directEnv, cell)
216466 c.line(dst & " = " & selfIdent)
217467 else:
218468 genFn("fn", clauses, "", env, c, dst)
@@ -273,37 +523,267 @@ proc genLet(bindings: Value, body: seq[Value], dst: string, env: Env, c: Ctx) =
273523 proc genLoop(bindings: Value, body: seq[Value], dst: string, env: Env, c: Ctx) =
274524 if bindings.isNil or bindings.kind != kVector or bindings.items.len mod 2 != 0:
275525 err("loop requires an even-sized binding vector")
276- let lenv = newEnv(env)
277- var idents: seq[string] = @[]
526+ var names: seq[string] = @[]
527+ var inits: seq[Value] = @[]
278528 var i = 0
279529 while i < bindings.items.len:
280- let nm = symName(bindings.items[i])
281- let v = genExpr(bindings.items[i + 1], lenv, c)
282- let id = c.gensym("l" & mangle(nm))
283- c.line("var " & id & ": Value = " & v)
284- lenv.locals[nm] = id
285- idents.add id
530+ names.add symName(bindings.items[i])
531+ inits.add bindings.items[i + 1]
286532 i += 2
533+
534+ # Which loop variables can be held as raw int64? A variable qualifies when
535+ # its initialiser is provably an integer and so is every recur value for its
536+ # slot. Those recur values usually mention the loop variables themselves, so
537+ # start optimistic and demote until the set stops shrinking.
538+ var recurs: seq[seq[Value]] = @[]
539+ collectRecurs(body, recurs)
540+ for r in recurs:
541+ if r.len != names.len: recurs = @[]; break # arity error, reported later
542+ var isInt: seq[bool] = @[]
543+ for n in names: isInt.add true
544+ var probeIdents: seq[string] = @[]
545+ for n in names: probeIdents.add "probe"
546+ while true:
547+ let probe = newEnv(env)
548+ for j, n in names:
549+ probe.locals[n] = probeIdents[j]
550+ if isInt[j]: probe.ints.incl n
551+ var changed = false
552+ for j, n in names:
553+ if not isInt[j]: continue
554+ # an initialiser only sees the bindings before it, as in let
555+ let ienv = newEnv(env)
556+ for k in 0 ..< j:
557+ ienv.locals[names[k]] = probeIdents[k]
558+ if isInt[k]: ienv.ints.incl names[k]
559+ if intExpr(inits[j], ienv, c).len == 0:
560+ isInt[j] = false; changed = true; continue
561+ for r in recurs:
562+ if intExpr(r[j], probe, c).len == 0:
563+ isInt[j] = false; changed = true; break
564+ if not changed: break
565+
566+ let lenv = newEnv(env)
567+ var idents: seq[string] = @[]
568+ for j, n in names:
569+ let id = c.gensym("l" & mangle(n))
570+ if isInt[j]:
571+ c.line("var " & id & ": int64 = " & intExpr(inits[j], lenv, c))
572+ else:
573+ c.line("var " & id & ": Value = " & genExpr(inits[j], lenv, c))
574+ lenv.locals[n] = id
575+ if isInt[j]: lenv.ints.incl n
576+ idents.add id
287577 c.line("while true:")
288578 c.push
289579 c.recurStack.add idents
580+ c.recurInts.add isInt
290581 genBody(body, dst, lenv, c)
291582 discard c.recurStack.pop
583+ discard c.recurInts.pop
292584 c.line("break")
293585 c.pop
294586
295587 proc genCall(f: Value, args: seq[Value], dst: string, env: Env, c: Ctx) =
296- let fv = genExpr(f, env, c)
588+ # A call to a fn whose arity is known here becomes a direct Nim call: no
589+ # argument seq, no closure dispatch. When the target came from `def` the
590+ # name can still be rebound at runtime, so guard on the var cell.
591+ if f.kind == kSymbol:
592+ let d = lookupDirect(env, f.s, args.len)
593+ if d.prc.len > 0:
594+ var argIdents: seq[string] = @[]
595+ for a in args: argIdents.add genExprTemp(a, env, c)
596+ let direct = d.prc & "(" & argIdents.join(", ") & ")"
597+ if d.cell.len == 0:
598+ c.line(dst & " = " & direct)
599+ else:
600+ c.line("if cellIs(" & d.cell & ", " & d.fnVal & "):")
601+ c.push; c.line(dst & " = " & direct); c.pop
602+ c.line("else:")
603+ c.push
604+ c.line(dst & " = call(cellGet(" & d.cell & "), " &
605+ (if argIdents.len == 0: "emptyArgs" else: "@[" & argIdents.join(", ") & "]") & ")")
606+ c.pop
607+ return
608+ let key = f.s & "/" & $args.len
609+ if intrinsics.hasKey(key) and env.lookup(f.s).len == 0 and
610+ c.primStable(f.s):
611+ var argIdents: seq[string] = @[]
612+ for a in args: argIdents.add genExprTemp(a, env, c)
613+ c.line(dst & " = " & intrinsics[key] & "(" & argIdents.join(", ") & ")")
614+ return
615+ if intrinsics.hasKey(key) and env.lookup(f.s).len == 0:
616+ var argIdents: seq[string] = @[]
617+ for a in args: argIdents.add genExprTemp(a, env, c)
618+ let cell = c.cellFor(f.s)
619+ let k = c.coreFor(f.s)
620+ c.line("if cellIs(" & cell & ", " & k & "):")
621+ c.push
622+ c.line(dst & " = " & intrinsics[key] & "(" & argIdents.join(", ") & ")")
623+ c.pop
624+ c.line("else:")
625+ c.push
626+ c.line(dst & " = call(cellGet(" & cell & "), @[" & argIdents.join(", ") & "])")
627+ c.pop
628+ return
629+ let fv = genExprTemp(f, env, c)
297630 var argIdents: seq[string] = @[]
298- for a in args: argIdents.add genExpr(a, env, c)
631+ for a in args: argIdents.add genExprTemp(a, env, c)
299632 if argIdents.len == 0:
300633 c.line(dst & " = call(" & fv & ", emptyArgs)")
301634 else:
302635 c.line(dst & " = call(" & fv & ", @[" & argIdents.join(", ") & "])")
303636
637+## ------------------------------------------------------- int specialisation
638+##
639+## The remaining cost of a numeric loop is that every intermediate integer is a
640+## 24-byte Value moving through memory. These two compile a form straight to a
641+## Nim int64 or bool expression when that is provably what it yields, so the C
642+## compiler sees an ordinary integer loop and can keep it in registers.
643+##
644+## "Provably" leans on primStable: with no eval, no defmacro and no def of the
645+## name anywhere in the program, `+` is arithmetic for the life of the process,
646+## so no runtime guard is needed on this path.
647+
648+proc intExpr(f: Value, env: Env, c: Ctx): string =
649+ ## A Nim int64 expression, or "" when the form is not provably an integer.
650+ case f.kind
651+ of kInt:
652+ "int64(" & $f.i & ")"
653+ of kSymbol:
654+ if env.isIntLocal(f.s): env.lookup(f.s) else: ""
655+ of kList:
656+ if f.items.len == 0: return ""
657+ let head = f.items[0]
658+ if head.kind != kSymbol: return ""
659+ let args = f.items[1 .. ^1]
660+ # (if c a b) is an int when both arms are
661+ if head.s == "if" and args.len == 3:
662+ let cond = boolExpr(args[0], env, c)
663+ if cond.len == 0: return ""
664+ let a = intExpr(args[1], env, c)
665+ if a.len == 0: return ""
666+ let b = intExpr(args[2], env, c)
667+ if b.len == 0: return ""
668+ return "(if " & cond & ": " & a & " else: " & b & ")"
669+ if specialHeads.contains(head.s): return ""
670+ if env.lookup(head.s).len == 0 and c.primStable(head.s):
671+ if args.len == 2 and (intOps.hasKey(head.s) or intCalls.hasKey(head.s)):
672+ let a = intExpr(args[0], env, c)
673+ if a.len == 0: return ""
674+ let b = intExpr(args[1], env, c)
675+ if b.len == 0: return ""
676+ if intCalls.hasKey(head.s):
677+ return intCalls[head.s] & "(" & a & ", " & b & ")"
678+ return "(" & a & " " & intOps[head.s] & " " & b & ")"
679+ if args.len == 1 and (head.s == "inc" or head.s == "dec"):
680+ let a = intExpr(args[0], env, c)
681+ if a.len == 0: return ""
682+ return "(" & a & (if head.s == "inc": " + 1" else: " - 1") & ")"
683+ # a call to an int-specialised fn that returns a raw int64
684+ let (prc, boxes) = env.lookupIntFn(head.s, args.len)
685+ if prc.len > 0 and not boxes:
686+ var ids: seq[string] = @[]
687+ for a in args:
688+ let e = intExpr(a, env, c)
689+ if e.len == 0: return ""
690+ ids.add e
691+ return prc & "(" & ids.join(", ") & ")"
692+ ""
693+ else:
694+ ""
695+
696+proc boolExpr(f: Value, env: Env, c: Ctx): string =
697+ ## A Nim bool expression for a comparison between provable integers.
698+ if f.kind != kList or f.items.len != 3: return ""
699+ let head = f.items[0]
700+ if head.kind != kSymbol or not cmpOps.hasKey(head.s): return ""
701+ if env.lookup(head.s).len > 0 or not c.primStable(head.s): return ""
702+ let a = intExpr(f.items[1], env, c)
703+ if a.len == 0: return ""
704+ let b = intExpr(f.items[2], env, c)
705+ if b.len == 0: return ""
706+ "(" & a & " " & cmpOps[head.s] & " " & b & ")"
707+
708+proc tryExprs(xs: seq[Value], env: Env, c: Ctx, ids: var seq[string]): bool =
709+ ## All-or-nothing: if any subform needs statements, the caller must fall back
710+ ## for every one of them, or an earlier operand could be read after a later
711+ ## operand's statements have run.
712+ for x in xs:
713+ let e = tryExpr(x, env, c)
714+ if e.len == 0: return false
715+ ids.add e
716+ true
717+
718+proc tryExpr(f: Value, env: Env, c: Ctx): string =
719+ ## Compile a form to a single Nim expression, or "" if it needs statements.
720+ ## Keeping a subexpression as an expression is what lets the C compiler hold
721+ ## it in a register instead of round-tripping it through a Value slot.
722+ case f.kind
723+ of kNil, kBool, kInt, kFloat, kStr, kKeyword:
724+ quoteLit(f)
725+ of kSymbol:
726+ let local = env.lookup(f.s)
727+ if local.len == 0: return "cellGet(" & c.cellFor(f.s) & ")"
728+ if env.isIntLocal(f.s): "mkInt(" & local & ")" else: local
729+ of kVector, kSet:
730+ var ids: seq[string] = @[]
731+ if not tryExprs(f.items, env, c, ids): return ""
732+ (if f.kind == kVector: "mkVector(" else: "mkSet(") &
733+ (if ids.len == 0: "newSeq[Value]()" else: "@[" & ids.join(", ") & "]") & ")"
734+ of kMap:
735+ var parts: seq[string] = @[]
736+ for (k, v) in f.pairs:
737+ let ke = tryExpr(k, env, c)
738+ if ke.len == 0: return ""
739+ let ve = tryExpr(v, env, c)
740+ if ve.len == 0: return ""
741+ parts.add "(" & ke & ", " & ve & ")"
742+ "mkMap(" & (if parts.len == 0: "newSeq[(Value, Value)]()"
743+ else: "@[" & parts.join(", ") & "]") & ")"
744+ of kList:
745+ if f.items.len == 0: return "mkList(newSeq[Value]())"
746+ let head = f.items[0]
747+ let args = f.items[1 .. ^1]
748+ if head.kind == kSymbol and specialHeads.contains(head.s): return ""
749+ var ids: seq[string] = @[]
750+ if not tryExprs(args, env, c, ids): return ""
751+ if head.kind == kSymbol:
752+ let (iprc, iboxes) = env.lookupIntFn(head.s, args.len)
753+ if iprc.len > 0:
754+ var iids: seq[string] = @[]
755+ var ok = true
756+ for a in args:
757+ let e = intExpr(a, env, c)
758+ if e.len == 0: ok = false; break
759+ iids.add e
760+ if ok:
761+ let callI = iprc & "(" & iids.join(", ") & ")"
762+ return (if iboxes: callI else: "mkInt(" & callI & ")")
763+ let d = lookupDirect(env, head.s, args.len)
764+ if d.prc.len > 0 and (d.cell.len == 0 or c.fnStable(head.s)):
765+ # a self-call, or a name only one def form ever targets
766+ return d.prc & "(" & ids.join(", ") & ")"
767+ if d.prc.len == 0:
768+ let key = head.s & "/" & $args.len
769+ if intrinsics.hasKey(key) and env.lookup(head.s).len == 0:
770+ if c.primStable(head.s):
771+ return intrinsics[key] & "(" & ids.join(", ") & ")"
772+ return intrinsics[key] & "g(" & c.cellFor(head.s) & ", " &
773+ c.coreFor(head.s) & ", " & ids.join(", ") & ")"
774+ let hv = tryExpr(head, env, c)
775+ if hv.len == 0: return ""
776+ "call(" & hv & ", " &
777+ (if ids.len == 0: "emptyArgs" else: "@[" & ids.join(", ") & "]") & ")"
778+ of kFn, kCons, kLazy:
779+ ""
780+
304781 proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
305782 if f.isNil:
306783 c.line(dst & " = NilV"); return
784+ let e = tryExpr(f, env, c)
785+ if e.len > 0:
786+ c.line(dst & " = " & e); return
307787 case f.kind
308788 of kNil, kBool, kInt, kFloat, kStr, kKeyword:
309789 c.line(dst & " = " & quoteLit(f))
@@ -313,19 +793,19 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
313793 else: c.line(dst & " = cellGet(" & c.cellFor(f.s) & ")")
314794 of kVector:
315795 var ids: seq[string] = @[]
316- for x in f.items: ids.add genExpr(x, env, c)
796+ for x in f.items: ids.add genExprTemp(x, env, c)
317797 c.line(dst & " = mkVector(" &
318798 (if ids.len == 0: "newSeq[Value]()" else: "@[" & ids.join(", ") & "]") & ")")
319799 of kSet:
320800 var ids: seq[string] = @[]
321- for x in f.items: ids.add genExpr(x, env, c)
801+ for x in f.items: ids.add genExprTemp(x, env, c)
322802 c.line(dst & " = mkSet(" &
323803 (if ids.len == 0: "newSeq[Value]()" else: "@[" & ids.join(", ") & "]") & ")")
324804 of kMap:
325805 var parts: seq[string] = @[]
326806 for (k, v) in f.pairs:
327- let ki = genExpr(k, env, c)
328- let vi = genExpr(v, env, c)
807+ let ki = genExprTemp(k, env, c)
808+ let vi = genExprTemp(v, env, c)
329809 parts.add "(" & ki & ", " & vi & ")"
330810 c.line(dst & " = mkMap(" &
331811 (if parts.len == 0: "newSeq[(Value, Value)]()" else: "@[" & parts.join(", ") & "]") & ")")
@@ -345,8 +825,7 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
345825 return
346826 of "if":
347827 if args.len < 2: err("Too few arguments to if")
348- let cv = genExpr(args[0], env, c)
349- c.line("if truthy(" & cv & "):")
828+ c.line("if " & genCond(args[0], env, c) & ":")
350829 c.push; genInto(args[1], dst, env, c); c.pop
351830 c.line("else:")
352831 c.push
@@ -371,8 +850,16 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
371850 if targets.len != args.len:
372851 err("Mismatched argument count to recur: expected " & $targets.len &
373852 ", got " & $args.len)
853+ let targetInts = c.recurInts[^1]
374854 var tmps: seq[string] = @[]
375- for a in args: tmps.add genExpr(a, env, c)
855+ for i, a in args:
856+ if targetInts[i]:
857+ let e = intExpr(a, env, c)
858+ let t = c.gensym("t")
859+ c.line("let " & t & ": int64 = " & e)
860+ tmps.add t
861+ else:
862+ tmps.add genExprTemp(a, env, c)
376863 for i, t in tmps: c.line(targets[i] & " = " & t)
377864 c.line("continue")
378865 return
@@ -400,7 +887,7 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
400887 if rest.len > 0 and rest[0].kind == kMap: rest = rest[1 .. ^1] # attr map
401888 let fv = c.gensym("fn")
402889 c.line("var " & fv & ": Value = NilV")
403- genFnForm(rest, env, c, fv, nm)
890+ genFnForm(rest, env, c, fv, nm, env, c.cellFor(nm))
404891 c.line(dst & " = setVar(" & nimStr(nm) & ", " & fv & ")")
405892 return
406893 of "defmacro":
@@ -429,22 +916,19 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
429916 return
430917 of "when":
431918 if args.len == 0: err("when requires a test")
432- let cv = genExpr(args[0], env, c)
433- c.line("if truthy(" & cv & "):")
919+ c.line("if " & genCond(args[0], env, c) & ":")
434920 c.push; genBody(args[1 .. ^1], dst, env, c); c.pop
435921 c.line("else:")
436922 c.push; c.line(dst & " = NilV"); c.pop
437923 return
438924 of "when-not":
439- let cv = genExpr(args[0], env, c)
440- c.line("if not truthy(" & cv & "):")
925+ c.line("if not (" & genCond(args[0], env, c) & "):")
441926 c.push; genBody(args[1 .. ^1], dst, env, c); c.pop
442927 c.line("else:")
443928 c.push; c.line(dst & " = NilV"); c.pop
444929 return
445930 of "if-not":
446- let cv = genExpr(args[0], env, c)
447- c.line("if not truthy(" & cv & "):")
931+ c.line("if not (" & genCond(args[0], env, c) & "):")
448932 c.push; genInto(args[1], dst, env, c); c.pop
449933 c.line("else:")
450934 c.push
@@ -460,8 +944,7 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
460944 if isSym(args[i], "else") or (args[i].kind == kKeyword and args[i].s == "else"):
461945 genInto(args[i + 1], dst, env, c)
462946 break
463- let cv = genExpr(args[i], env, c)
464- c.line("if truthy(" & cv & "):")
947+ c.line("if " & genCond(args[i], env, c) & ":")
465948 c.push
466949 genInto(args[i + 1], dst, env, c)
467950 c.pop
@@ -474,12 +957,11 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
474957 let b = args[0]
475958 if b.kind != kVector or b.items.len != 2: err(head.s & " requires [sym test]")
476959 let nm = symName(b.items[0])
477- let tv = genExpr(b.items[1], env, c)
478- c.line("if truthy(" & tv & "):")
960+ let id = c.gensym("l" & mangle(nm))
961+ c.line("var " & id & ": Value = " & genExpr(b.items[1], env, c))
962+ c.line("if truthy(" & id & "):")
479963 c.push
480964 let benv = newEnv(env)
481- let id = c.gensym("l" & mangle(nm))
482- c.line("var " & id & ": Value = " & tv)
483965 benv.locals[nm] = id
484966 if head.s == "when-let": genBody(args[1 .. ^1], dst, benv, c)
485967 else: genInto(args[1], dst, benv, c)
@@ -595,10 +1077,26 @@ import runtime, core
5951077 proc cljMain() =
5961078 """
5971079
1080+proc collectDefs(f: Value, into: var CountTable[string]) =
1081+ ## Every name this program can rebind at runtime. `def` and `defn` are the
1082+ ## only paths to setVar, and clonim has no eval, no defmacro and no intern,
1083+ ## so a name that no def form targets holds whatever registerCore gave it for
1084+ ## the life of the process. That is what lets call sites drop the cell guard
1085+ ## and lets the analyzer trust `+` to be arithmetic.
1086+ if f.isNil or f.kind != kList or f.items.len == 0: return
1087+ let head = f.items[0]
1088+ if head.kind == kSymbol and head.s in ["def", "defn", "defn-"] and
1089+ f.items.len > 1 and f.items[1].kind == kSymbol:
1090+ into.inc f.items[1].s
1091+ for x in f.items: collectDefs(x, into)
1092+
5981093 proc compileForms*(forms: seq[Value]): string =
5991094 let c = Ctx(body: @[], indent: 1, counter: 0, recurStack: @[],
1095+ recurInts: @[], cores: initTable[string, string](),
6001096 defined: initHashSet[string](), prelude: @[],
601- cells: initTable[string, string]())
1097+ cells: initTable[string, string](),
1098+ defCounts: initCountTable[string]())
1099+ for f in forms: collectDefs(f, c.defCounts)
6021100 let env = newEnv()
6031101 for f in forms:
6041102 let t = c.gensym("top")
@@ -7,21 +7,37 @@ import std/[tables, strutils, sets]
7 import runtime, reader7 import runtime, reader
8 8
9 type9 type
10+ ## A fn whose arity is known at the call site, so it can be reached as a
11+ ## plain Nim proc instead of through a seq of boxed args.
12+ Direct = object
13+ prc: string # nim proc ident taking positional Values
14+ cell: string # var cell to guard on ("" = no guard)
15+ fnVal: string # nim ident of the fn Value to compare
16+
10 Env = ref object17 Env = ref object
11 parent: Env18 parent: Env
12 locals: Table[string, string] # clojure name -> nim identifier19 locals: Table[string, string] # clojure name -> nim identifier
20+ ints: HashSet[string] # of those, the ones held as raw int64
21+ directs: Table[string, Direct] # "name/arity" -> positional entry point
22+ intFns: Table[string, string] # "name/arity" -> int-specialised proc
23+ intFnBoxes: HashSet[string] # of those, the ones returning Value
13 24
14 Ctx = ref object25 Ctx = ref object
15 body: seq[string] # emitted lines26 body: seq[string] # emitted lines
16 indent: int27 indent: int
17 counter: int28 counter: int
18 recurStack: seq[seq[string]] # nim idents of the enclosing recur target29 recurStack: seq[seq[string]] # nim idents of the enclosing recur target
30+ recurInts: seq[seq[bool]] # which of those are raw int64
19 defined: HashSet[string] # names def'd so far (for nicer errors)31 defined: HashSet[string] # names def'd so far (for nicer errors)
20 prelude: seq[string] # hoisted var-cell resolutions32 prelude: seq[string] # hoisted var-cell resolutions
21 cells: Table[string, string] # clojure var name -> nim cell ident33 cells: Table[string, string] # clojure var name -> nim cell ident
34+ cores: Table[string, string] # var name -> nim ident holding its core fn
35+ defCounts: CountTable[string] # how many def forms target each name
22 36
23 proc newEnv(parent: Env = nil): Env =37 proc newEnv(parent: Env = nil): Env =
24- Env(parent: parent, locals: initTable[string, string]())38+ Env(parent: parent, locals: initTable[string, string](),
39+ ints: initHashSet[string](), directs: initTable[string, Direct](),
40+ intFns: initTable[string, string](), intFnBoxes: initHashSet[string]())
25 41
26 proc lookup(env: Env, name: string): string =42 proc lookup(env: Env, name: string): string =
27 var e = env43 var e = env
@@ -30,6 +46,45 @@ proc lookup(env: Env, name: string): string =
30 e = e.parent46 e = e.parent
31 ""47 ""
32 48
49+proc isIntLocal(env: Env, name: string): bool =
50+ ## True when the name's Nim binding is an int64 rather than a Value.
51+ var e = env
52+ while e != nil:
53+ if e.locals.hasKey(name): return e.ints.contains(name)
54+ e = e.parent
55+ false
56+
57+proc lookupIntFn(env: Env, name: string, arity: int): (string, bool) =
58+ ## The int-specialised entry point for a fn, and whether it returns a Value
59+ ## (true) or a raw int64 (false).
60+ let key = name & "/" & $arity
61+ var e = env
62+ while e != nil:
63+ if e.intFns.hasKey(key): return (e.intFns[key], e.intFnBoxes.contains(key))
64+ if e.locals.hasKey(name): return ("", false)
65+ e = e.parent
66+ ("", false)
67+
68+proc lookupDirect(env: Env, name: string, arity: int): Direct =
69+ ## A local binding of the same name shadows the direct entry point.
70+ let key = name & "/" & $arity
71+ var e = env
72+ while e != nil:
73+ if e.directs.hasKey(key): return e.directs[key]
74+ if e.locals.hasKey(name): return Direct()
75+ e = e.parent
76+ Direct()
77+
78+proc primStable(c: Ctx, name: string): bool =
79+ ## A core builtin still holds its registerCore value everywhere: no def in
80+ ## this program targets the name at all.
81+ c.defCounts[name] == 0
82+
83+proc fnStable(c: Ctx, name: string): bool =
84+ ## A user fn is reached by exactly one definition, so the one a call site was
85+ ## compiled against is the only one it can ever see.
86+ c.defCounts[name] <= 1
87+
33 proc line(c: Ctx, s: string) =88 proc line(c: Ctx, s: string) =
34 c.body.add repeat(" ", c.indent) & s89 c.body.add repeat(" ", c.indent) & s
35 90
@@ -70,6 +125,36 @@ proc cellFor(c: Ctx, name: string): string =
70 c.cells[name] = result125 c.cells[name] = result
71 c.prelude.add " let " & result & " = varCell(" & nimStr(name) & ")"126 c.prelude.add " let " & result & " = varCell(" & nimStr(name) & ")"
72 127
128+## Builtins with a cheap inline form. A call site at the matching arity emits
129+## the inline proc directly, guarded on the var still holding the core fn.
130+const intrinsics = {
131+ "+/2": "add2", "-/2": "sub2", "*/2": "mul2",
132+ "</2": "lt2", ">/2": "gt2", "<=/2": "le2", ">=/2": "ge2",
133+ "=/2": "eq2", "not=/2": "ne2", "inc/1": "inc1", "dec/1": "dec1",
134+}.toTable
135+
136+## Heads that compile to statements, never to a single expression.
137+const specialHeads = ["quote", "if", "do", "let", "let*", "loop", "loop*",
138+ "recur", "fn", "fn*", "def", "defn", "defn-", "defmacro", "and", "or",
139+ "when", "when-not", "if-not", "cond", "when-let", "if-let", "->", "->>",
140+ "doseq", "dotimes", "try", "comment", "ns", "require", "in-ns", "use",
141+ "import", "set!", "declare"].toHashSet
142+
143+const intOps = {"+": "+", "-": "-", "*": "*"}.toTable
144+const intCalls = {"quot": "idiv", "rem": "irem"}.toTable
145+const cmpOps = {"<": "<", ">": ">", "<=": "<=", ">=": ">=", "=": "==",
146+ "not=": "!="}.toTable
147+
148+proc coreFor(c: Ctx, name: string): string =
149+ ## The value a core builtin had at program start, captured once, so call
150+ ## sites can tell whether the var still holds it.
151+ if c.cores.hasKey(name): return c.cores[name]
152+ let cell = c.cellFor(name)
153+ inc c.counter
154+ result = "k_" & $c.counter
155+ c.cores[name] = result
156+ c.prelude.add " let " & result & " = cellGet(" & cell & ")"
157+
73 proc isSym(v: Value, name: string): bool =158 proc isSym(v: Value, name: string): bool =
74 not v.isNil and v.kind == kSymbol and v.s == name159 not v.isNil and v.kind == kSymbol and v.s == name
75 160
@@ -110,18 +195,53 @@ proc emptySeqFix(s: string, elemType: string): string =
110 195
111 # ------------------------------------------------------------- code gen196 # ------------------------------------------------------------- code gen
112 proc genInto(f: Value, dst: string, env: Env, c: Ctx)197 proc genInto(f: Value, dst: string, env: Env, c: Ctx)
198+proc tryExpr(f: Value, env: Env, c: Ctx): string
199+proc intExpr(f: Value, env: Env, c: Ctx): string
200+proc boolExpr(f: Value, env: Env, c: Ctx): string
113 201
114 proc genExpr(f: Value, env: Env, c: Ctx): string =202 proc genExpr(f: Value, env: Env, c: Ctx): string =
203+ ## A Nim expression denoting the form's value. Forms that compile to a single
204+ ## expression are returned as-is; the rest go through a temporary slot.
205+ result = tryExpr(f, env, c)
206+ if result.len > 0: return
115 result = c.gensym("t")207 result = c.gensym("t")
116 c.line("var " & result & ": Value = NilV")208 c.line("var " & result & ": Value = NilV")
117 genInto(f, result, env, c)209 genInto(f, result, env, c)
118 210
211+proc genExprTemp(f: Value, env: Env, c: Ctx): string =
212+ ## Like genExpr, but always materialises the value into a fresh local. Used
213+ ## where the value is read more than once, or must be computed before a
214+ ## later statement can overwrite what it reads.
215+ let e = tryExpr(f, env, c)
216+ if e.len > 0:
217+ result = c.gensym("t")
218+ c.line("let " & result & ": Value = " & e)
219+ return
220+ result = c.gensym("t")
221+ c.line("var " & result & ": Value = NilV")
222+ genInto(f, result, env, c)
223+
224+proc genCond(f: Value, env: Env, c: Ctx): string =
225+ ## A Nim bool for a test position. A comparison between provable integers
226+ ## becomes a machine compare; anything else falls back to truthy() on a Value.
227+ result = boolExpr(f, env, c)
228+ if result.len > 0: return
229+ result = "truthy(" & genExpr(f, env, c) & ")"
230+
231+proc genStmt(f: Value, env: Env, c: Ctx) =
232+ ## A form evaluated only for its effects.
233+ let e = tryExpr(f, env, c)
234+ if e.len > 0:
235+ c.line("discard " & e)
236+ return
237+ discard genExpr(f, env, c)
238+
119 proc genBody(forms: seq[Value], dst: string, env: Env, c: Ctx) =239 proc genBody(forms: seq[Value], dst: string, env: Env, c: Ctx) =
120 if forms.len == 0:240 if forms.len == 0:
121 c.line(dst & " = NilV")241 c.line(dst & " = NilV")
122 return242 return
123 for i in 0 ..< forms.len - 1:243 for i in 0 ..< forms.len - 1:
124- discard genExpr(forms[i], env, c)244+ genStmt(forms[i], env, c)
125 genInto(forms[^1], dst, env, c)245 genInto(forms[^1], dst, env, c)
126 246
127 type247 type
@@ -143,41 +263,170 @@ proc parseParams(v: Value): FnClause =
143 result.params.add symName(p)263 result.params.add symName(p)
144 inc i264 inc i
145 265
146-proc genFn(name: string, clauses: seq[FnClause], selfIdent: string, env: Env, c: Ctx, dst: string) =266+proc genClauseBody(cl: FnClause, name, selfIdent: string, env: Env, c: Ctx,
267+ bindTo: proc (i: int, p: string): string, res: string,
268+ selfDirect = "", intParams = false) =
269+ ## Shared between the positional proc and the generic dispatcher: bind the
270+ ## params to mutable locals (recur assigns them), then run the body.
271+ let fenv = newEnv(env)
272+ if selfIdent.len > 0 and name.len > 0:
273+ fenv.locals[name] = selfIdent
274+ # Registered alongside the self local, and checked first, so a self-call at
275+ # the matching arity reaches the positional proc rather than the fn Value.
276+ if selfDirect.len > 0:
277+ fenv.directs[name & "/" & $cl.params.len] = Direct(prc: selfDirect)
278+ var recurIdents: seq[string] = @[]
279+ for i, p in cl.params:
280+ let id = c.gensym("p" & mangle(p))
281+ c.line("var " & id & (if intParams: ": int64 = " else: ": Value = ") &
282+ bindTo(i, p))
283+ fenv.locals[p] = id
284+ if intParams: fenv.ints.incl p
285+ recurIdents.add id
286+ if cl.restParam.len > 0:
287+ let id = c.gensym("p" & mangle(cl.restParam))
288+ c.line("var " & id & ": Value = " & bindTo(-1, cl.restParam))
289+ fenv.locals[cl.restParam] = id
290+ c.line("var " & res & ": Value = NilV")
291+ c.line("while true:")
292+ c.push
293+ c.recurStack.add recurIdents
294+ var recurIsInt = newSeq[bool](recurIdents.len)
295+ if intParams:
296+ for j in 0 ..< recurIsInt.len: recurIsInt[j] = true
297+ c.recurInts.add recurIsInt
298+ genBody(cl.body, res, fenv, c)
299+ discard c.recurStack.pop
300+ discard c.recurInts.pop
301+ c.line("break")
302+ c.pop
303+
304+proc collectRecurs(forms: seq[Value], into: var seq[seq[Value]]) =
305+ ## The recur forms belonging to the innermost enclosing loop. Nested fn and
306+ ## loop forms establish their own recur target, so their bodies are skipped.
307+ for f in forms:
308+ if f.isNil or f.kind != kList or f.items.len == 0: continue
309+ let head = f.items[0]
310+ if head.kind == kSymbol:
311+ if head.s == "recur":
312+ into.add f.items[1 .. ^1]
313+ continue
314+ if head.s in ["loop", "loop*", "fn", "fn*", "defn", "defn-"]: continue
315+ collectRecurs(f.items, into)
316+
317+proc usesIntPrim(c: Ctx, forms: seq[Value]): bool =
318+ ## Whether an int-specialised twin could differ from the generic proc at all.
319+ ## Emitting one for a fn that never does arithmetic just doubles the work Nim
320+ ## has to do; this gate is an optimisation only, never a semantic decision.
321+ for f in forms:
322+ if f.isNil or f.kind != kList or f.items.len == 0: continue
323+ let head = f.items[0]
324+ if head.kind == kSymbol and c.primStable(head.s) and
325+ (intOps.hasKey(head.s) or intCalls.hasKey(head.s) or
326+ cmpOps.hasKey(head.s) or head.s == "inc" or head.s == "dec"):
327+ return true
328+ if usesIntPrim(c, f.items): return true
329+ false
330+
331+proc genFn(name: string, clauses: seq[FnClause], selfIdent: string, env: Env,
332+ c: Ctx, dst: string, directEnv: Env = nil, cell = "") =
333+ ## Each fixed-arity clause gets a real Nim proc taking its params
334+ ## positionally; the mkFn wrapper is just an arity dispatcher onto those, and
335+ ## call sites that know the arity skip the wrapper entirely.
336+ var directProcs: seq[string] = @[] # parallel to clauses, "" for variadic
337+ for cl in clauses:
338+ if cl.restParam.len > 0:
339+ directProcs.add ""
340+ continue
341+ let prc = c.gensym("uf" & mangle(if name.len > 0: name else: "fn"))
342+ var params: seq[string] = @[]
343+ for i in 0 ..< cl.params.len: params.add "a" & $i & ": Value"
344+ c.line("proc " & prc & "(" & params.join(", ") & "): Value =")
345+ c.push
346+ let res = c.gensym("res")
347+ # selfDirect makes a self-call at this arity a plain recursive Nim call.
348+ genClauseBody(cl, name, selfIdent, env, c,
349+ proc (i: int, p: string): string = "a" & $i, res, prc)
350+ c.line("return " & res)
351+ c.pop
352+ directProcs.add prc
353+ if directEnv != nil:
354+ directEnv.directs[name & "/" & $cl.params.len] =
355+ Direct(prc: prc, cell: cell, fnVal: dst)
356+
357+ # An int-specialised twin, so a caller with integer arguments never boxes
358+ # them. Emitted beside the generic proc rather than replacing it: callers
359+ # that cannot prove their arguments are integers still need the Value one.
360+ if name.len > 0 and cl.params.len > 0 and c.usesIntPrim(cl.body):
361+ let key = name & "/" & $cl.params.len
362+ let iprc = c.gensym("ufi" & mangle(name))
363+ # Does the body yield an integer? Probe with the parameters typed and the
364+ # fn optimistically assumed to return one, so self-recursion types too.
365+ # A recur inside the clause reassigns the parameters, so every recur
366+ # value has to stay integral too. Anything less and the twin is dropped
367+ # rather than emitted with a mix of int64 and Value parameters.
368+ let probe = newEnv(env)
369+ for i, p in cl.params:
370+ probe.locals[p] = "a" & $i
371+ probe.ints.incl p
372+ probe.intFns[key] = iprc
373+ var recurSafe = true
374+ var myRecurs: seq[seq[Value]] = @[]
375+ collectRecurs(cl.body, myRecurs)
376+ for r in myRecurs:
377+ if r.len != cl.params.len: recurSafe = false; break
378+ for a in r:
379+ if intExpr(a, probe, c).len == 0: recurSafe = false; break
380+ if not recurSafe: break
381+ var boxes = true
382+ if cl.body.len == 1 and intExpr(cl.body[0], probe, c).len > 0:
383+ boxes = false
384+ if recurSafe:
385+ var iparams: seq[string] = @[]
386+ for i in 0 ..< cl.params.len: iparams.add "a" & $i & ": int64"
387+ c.line("proc " & iprc & "(" & iparams.join(", ") & "): " &
388+ (if boxes: "Value" else: "int64") & " =")
389+ c.push
390+ let ienv = newEnv(env)
391+ ienv.intFns[key] = iprc
392+ if boxes: ienv.intFnBoxes.incl key
393+ if boxes:
394+ let ires = c.gensym("res")
395+ genClauseBody(cl, name, selfIdent, ienv, c,
396+ proc (i: int, p: string): string = "a" & $i, ires, "", true)
397+ c.line("return " & ires)
398+ else:
399+ for i, p in cl.params:
400+ ienv.locals[p] = "a" & $i
401+ ienv.ints.incl p
402+ c.line("return " & intExpr(cl.body[0], ienv, c))
403+ c.pop
404+ if directEnv != nil and c.fnStable(name):
405+ directEnv.intFns[key] = iprc
406+ if boxes: directEnv.intFnBoxes.incl key
407+
147 let argsIdent = c.gensym("args")408 let argsIdent = c.gensym("args")
148 c.line(dst & " = mkFn(" & nimStr(name) & ", proc (" & argsIdent & ": seq[Value]): Value =")409 c.line(dst & " = mkFn(" & nimStr(name) & ", proc (" & argsIdent & ": seq[Value]): Value =")
149 c.push410 c.push
150 var first = true411 var first = true
151- for cl in clauses:412+ for ci, cl in clauses:
152 let cond =413 let cond =
153 if cl.restParam.len > 0: argsIdent & ".len >= " & $cl.params.len414 if cl.restParam.len > 0: argsIdent & ".len >= " & $cl.params.len
154 else: argsIdent & ".len == " & $cl.params.len415 else: argsIdent & ".len == " & $cl.params.len
155 c.line((if first: "if " else: "elif ") & cond & ":")416 c.line((if first: "if " else: "elif ") & cond & ":")
156 first = false417 first = false
157 c.push418 c.push
158- let fenv = newEnv(env)419+ if directProcs[ci].len > 0:
159- if selfIdent.len > 0 and name.len > 0:420+ var fwd: seq[string] = @[]
160- fenv.locals[name] = selfIdent421+ for i in 0 ..< cl.params.len: fwd.add "argAt(" & argsIdent & ", " & $i & ")"
161- var recurIdents: seq[string] = @[]422+ c.line("return " & directProcs[ci] & "(" & fwd.join(", ") & ")")
162- for i, p in cl.params:423+ else:
163- let id = c.gensym("p" & mangle(p))424+ let res = c.gensym("res")
164- c.line("var " & id & ": Value = argAt(" & argsIdent & ", " & $i & ")")425+ genClauseBody(cl, name, selfIdent, env, c,
165- fenv.locals[p] = id426+ proc (i: int, p: string): string =
166- recurIdents.add id427+ if i < 0: "restArgs(" & argsIdent & ", " & $cl.params.len & ")"
167- if cl.restParam.len > 0:428+ else: "argAt(" & argsIdent & ", " & $i & ")", res)
168- let id = c.gensym("p" & mangle(cl.restParam))429+ c.line("return " & res)
169- c.line("var " & id & ": Value = restArgs(" & argsIdent & ", " & $cl.params.len & ")")
170- fenv.locals[cl.restParam] = id
171- let res = c.gensym("res")
172- c.line("var " & res & ": Value = NilV")
173- c.line("while true:")
174- c.push
175- c.recurStack.add recurIdents
176- genBody(cl.body, res, fenv, c)
177- discard c.recurStack.pop
178- c.line("break")
179- c.pop
180- c.line("return " & res)
181 c.pop430 c.pop
182 c.line("else:")431 c.line("else:")
183 c.push432 c.push
@@ -187,7 +436,8 @@ proc genFn(name: string, clauses: seq[FnClause], selfIdent: string, env: Env, c:
187 c.pop436 c.pop
188 c.line(")")437 c.line(")")
189 438
190-proc genFnForm(args: seq[Value], env: Env, c: Ctx, dst: string, defName: string) =439+proc genFnForm(args: seq[Value], env: Env, c: Ctx, dst: string, defName: string,
440+ directEnv: Env = nil, cell = "") =
191 ## (fn name? [params] body...) or (fn name? ([params] body...) ...)441 ## (fn name? [params] body...) or (fn name? ([params] body...) ...)
192 var i = 0442 var i = 0
193 var name = defName443 var name = defName
@@ -212,7 +462,7 @@ proc genFnForm(args: seq[Value], env: Env, c: Ctx, dst: string, defName: string)
212 # bind the fn to a local so it can recur by name462 # bind the fn to a local so it can recur by name
213 selfIdent = c.gensym("self" & mangle(name))463 selfIdent = c.gensym("self" & mangle(name))
214 c.line("var " & selfIdent & ": Value = NilV")464 c.line("var " & selfIdent & ": Value = NilV")
215- genFn(name, clauses, selfIdent, env, c, selfIdent)465+ genFn(name, clauses, selfIdent, env, c, selfIdent, directEnv, cell)
216 c.line(dst & " = " & selfIdent)466 c.line(dst & " = " & selfIdent)
217 else:467 else:
218 genFn("fn", clauses, "", env, c, dst)468 genFn("fn", clauses, "", env, c, dst)
@@ -273,37 +523,267 @@ proc genLet(bindings: Value, body: seq[Value], dst: string, env: Env, c: Ctx) =
273 proc genLoop(bindings: Value, body: seq[Value], dst: string, env: Env, c: Ctx) =523 proc genLoop(bindings: Value, body: seq[Value], dst: string, env: Env, c: Ctx) =
274 if bindings.isNil or bindings.kind != kVector or bindings.items.len mod 2 != 0:524 if bindings.isNil or bindings.kind != kVector or bindings.items.len mod 2 != 0:
275 err("loop requires an even-sized binding vector")525 err("loop requires an even-sized binding vector")
276- let lenv = newEnv(env)526+ var names: seq[string] = @[]
277- var idents: seq[string] = @[]527+ var inits: seq[Value] = @[]
278 var i = 0528 var i = 0
279 while i < bindings.items.len:529 while i < bindings.items.len:
280- let nm = symName(bindings.items[i])530+ names.add symName(bindings.items[i])
281- let v = genExpr(bindings.items[i + 1], lenv, c)531+ inits.add bindings.items[i + 1]
282- let id = c.gensym("l" & mangle(nm))
283- c.line("var " & id & ": Value = " & v)
284- lenv.locals[nm] = id
285- idents.add id
286 i += 2532 i += 2
533+
534+ # Which loop variables can be held as raw int64? A variable qualifies when
535+ # its initialiser is provably an integer and so is every recur value for its
536+ # slot. Those recur values usually mention the loop variables themselves, so
537+ # start optimistic and demote until the set stops shrinking.
538+ var recurs: seq[seq[Value]] = @[]
539+ collectRecurs(body, recurs)
540+ for r in recurs:
541+ if r.len != names.len: recurs = @[]; break # arity error, reported later
542+ var isInt: seq[bool] = @[]
543+ for n in names: isInt.add true
544+ var probeIdents: seq[string] = @[]
545+ for n in names: probeIdents.add "probe"
546+ while true:
547+ let probe = newEnv(env)
548+ for j, n in names:
549+ probe.locals[n] = probeIdents[j]
550+ if isInt[j]: probe.ints.incl n
551+ var changed = false
552+ for j, n in names:
553+ if not isInt[j]: continue
554+ # an initialiser only sees the bindings before it, as in let
555+ let ienv = newEnv(env)
556+ for k in 0 ..< j:
557+ ienv.locals[names[k]] = probeIdents[k]
558+ if isInt[k]: ienv.ints.incl names[k]
559+ if intExpr(inits[j], ienv, c).len == 0:
560+ isInt[j] = false; changed = true; continue
561+ for r in recurs:
562+ if intExpr(r[j], probe, c).len == 0:
563+ isInt[j] = false; changed = true; break
564+ if not changed: break
565+
566+ let lenv = newEnv(env)
567+ var idents: seq[string] = @[]
568+ for j, n in names:
569+ let id = c.gensym("l" & mangle(n))
570+ if isInt[j]:
571+ c.line("var " & id & ": int64 = " & intExpr(inits[j], lenv, c))
572+ else:
573+ c.line("var " & id & ": Value = " & genExpr(inits[j], lenv, c))
574+ lenv.locals[n] = id
575+ if isInt[j]: lenv.ints.incl n
576+ idents.add id
287 c.line("while true:")577 c.line("while true:")
288 c.push578 c.push
289 c.recurStack.add idents579 c.recurStack.add idents
580+ c.recurInts.add isInt
290 genBody(body, dst, lenv, c)581 genBody(body, dst, lenv, c)
291 discard c.recurStack.pop582 discard c.recurStack.pop
583+ discard c.recurInts.pop
292 c.line("break")584 c.line("break")
293 c.pop585 c.pop
294 586
295 proc genCall(f: Value, args: seq[Value], dst: string, env: Env, c: Ctx) =587 proc genCall(f: Value, args: seq[Value], dst: string, env: Env, c: Ctx) =
296- let fv = genExpr(f, env, c)588+ # A call to a fn whose arity is known here becomes a direct Nim call: no
589+ # argument seq, no closure dispatch. When the target came from `def` the
590+ # name can still be rebound at runtime, so guard on the var cell.
591+ if f.kind == kSymbol:
592+ let d = lookupDirect(env, f.s, args.len)
593+ if d.prc.len > 0:
594+ var argIdents: seq[string] = @[]
595+ for a in args: argIdents.add genExprTemp(a, env, c)
596+ let direct = d.prc & "(" & argIdents.join(", ") & ")"
597+ if d.cell.len == 0:
598+ c.line(dst & " = " & direct)
599+ else:
600+ c.line("if cellIs(" & d.cell & ", " & d.fnVal & "):")
601+ c.push; c.line(dst & " = " & direct); c.pop
602+ c.line("else:")
603+ c.push
604+ c.line(dst & " = call(cellGet(" & d.cell & "), " &
605+ (if argIdents.len == 0: "emptyArgs" else: "@[" & argIdents.join(", ") & "]") & ")")
606+ c.pop
607+ return
608+ let key = f.s & "/" & $args.len
609+ if intrinsics.hasKey(key) and env.lookup(f.s).len == 0 and
610+ c.primStable(f.s):
611+ var argIdents: seq[string] = @[]
612+ for a in args: argIdents.add genExprTemp(a, env, c)
613+ c.line(dst & " = " & intrinsics[key] & "(" & argIdents.join(", ") & ")")
614+ return
615+ if intrinsics.hasKey(key) and env.lookup(f.s).len == 0:
616+ var argIdents: seq[string] = @[]
617+ for a in args: argIdents.add genExprTemp(a, env, c)
618+ let cell = c.cellFor(f.s)
619+ let k = c.coreFor(f.s)
620+ c.line("if cellIs(" & cell & ", " & k & "):")
621+ c.push
622+ c.line(dst & " = " & intrinsics[key] & "(" & argIdents.join(", ") & ")")
623+ c.pop
624+ c.line("else:")
625+ c.push
626+ c.line(dst & " = call(cellGet(" & cell & "), @[" & argIdents.join(", ") & "])")
627+ c.pop
628+ return
629+ let fv = genExprTemp(f, env, c)
297 var argIdents: seq[string] = @[]630 var argIdents: seq[string] = @[]
298- for a in args: argIdents.add genExpr(a, env, c)631+ for a in args: argIdents.add genExprTemp(a, env, c)
299 if argIdents.len == 0:632 if argIdents.len == 0:
300 c.line(dst & " = call(" & fv & ", emptyArgs)")633 c.line(dst & " = call(" & fv & ", emptyArgs)")
301 else:634 else:
302 c.line(dst & " = call(" & fv & ", @[" & argIdents.join(", ") & "])")635 c.line(dst & " = call(" & fv & ", @[" & argIdents.join(", ") & "])")
303 636
637+## ------------------------------------------------------- int specialisation
638+##
639+## The remaining cost of a numeric loop is that every intermediate integer is a
640+## 24-byte Value moving through memory. These two compile a form straight to a
641+## Nim int64 or bool expression when that is provably what it yields, so the C
642+## compiler sees an ordinary integer loop and can keep it in registers.
643+##
644+## "Provably" leans on primStable: with no eval, no defmacro and no def of the
645+## name anywhere in the program, `+` is arithmetic for the life of the process,
646+## so no runtime guard is needed on this path.
647+
648+proc intExpr(f: Value, env: Env, c: Ctx): string =
649+ ## A Nim int64 expression, or "" when the form is not provably an integer.
650+ case f.kind
651+ of kInt:
652+ "int64(" & $f.i & ")"
653+ of kSymbol:
654+ if env.isIntLocal(f.s): env.lookup(f.s) else: ""
655+ of kList:
656+ if f.items.len == 0: return ""
657+ let head = f.items[0]
658+ if head.kind != kSymbol: return ""
659+ let args = f.items[1 .. ^1]
660+ # (if c a b) is an int when both arms are
661+ if head.s == "if" and args.len == 3:
662+ let cond = boolExpr(args[0], env, c)
663+ if cond.len == 0: return ""
664+ let a = intExpr(args[1], env, c)
665+ if a.len == 0: return ""
666+ let b = intExpr(args[2], env, c)
667+ if b.len == 0: return ""
668+ return "(if " & cond & ": " & a & " else: " & b & ")"
669+ if specialHeads.contains(head.s): return ""
670+ if env.lookup(head.s).len == 0 and c.primStable(head.s):
671+ if args.len == 2 and (intOps.hasKey(head.s) or intCalls.hasKey(head.s)):
672+ let a = intExpr(args[0], env, c)
673+ if a.len == 0: return ""
674+ let b = intExpr(args[1], env, c)
675+ if b.len == 0: return ""
676+ if intCalls.hasKey(head.s):
677+ return intCalls[head.s] & "(" & a & ", " & b & ")"
678+ return "(" & a & " " & intOps[head.s] & " " & b & ")"
679+ if args.len == 1 and (head.s == "inc" or head.s == "dec"):
680+ let a = intExpr(args[0], env, c)
681+ if a.len == 0: return ""
682+ return "(" & a & (if head.s == "inc": " + 1" else: " - 1") & ")"
683+ # a call to an int-specialised fn that returns a raw int64
684+ let (prc, boxes) = env.lookupIntFn(head.s, args.len)
685+ if prc.len > 0 and not boxes:
686+ var ids: seq[string] = @[]
687+ for a in args:
688+ let e = intExpr(a, env, c)
689+ if e.len == 0: return ""
690+ ids.add e
691+ return prc & "(" & ids.join(", ") & ")"
692+ ""
693+ else:
694+ ""
695+
696+proc boolExpr(f: Value, env: Env, c: Ctx): string =
697+ ## A Nim bool expression for a comparison between provable integers.
698+ if f.kind != kList or f.items.len != 3: return ""
699+ let head = f.items[0]
700+ if head.kind != kSymbol or not cmpOps.hasKey(head.s): return ""
701+ if env.lookup(head.s).len > 0 or not c.primStable(head.s): return ""
702+ let a = intExpr(f.items[1], env, c)
703+ if a.len == 0: return ""
704+ let b = intExpr(f.items[2], env, c)
705+ if b.len == 0: return ""
706+ "(" & a & " " & cmpOps[head.s] & " " & b & ")"
707+
708+proc tryExprs(xs: seq[Value], env: Env, c: Ctx, ids: var seq[string]): bool =
709+ ## All-or-nothing: if any subform needs statements, the caller must fall back
710+ ## for every one of them, or an earlier operand could be read after a later
711+ ## operand's statements have run.
712+ for x in xs:
713+ let e = tryExpr(x, env, c)
714+ if e.len == 0: return false
715+ ids.add e
716+ true
717+
718+proc tryExpr(f: Value, env: Env, c: Ctx): string =
719+ ## Compile a form to a single Nim expression, or "" if it needs statements.
720+ ## Keeping a subexpression as an expression is what lets the C compiler hold
721+ ## it in a register instead of round-tripping it through a Value slot.
722+ case f.kind
723+ of kNil, kBool, kInt, kFloat, kStr, kKeyword:
724+ quoteLit(f)
725+ of kSymbol:
726+ let local = env.lookup(f.s)
727+ if local.len == 0: return "cellGet(" & c.cellFor(f.s) & ")"
728+ if env.isIntLocal(f.s): "mkInt(" & local & ")" else: local
729+ of kVector, kSet:
730+ var ids: seq[string] = @[]
731+ if not tryExprs(f.items, env, c, ids): return ""
732+ (if f.kind == kVector: "mkVector(" else: "mkSet(") &
733+ (if ids.len == 0: "newSeq[Value]()" else: "@[" & ids.join(", ") & "]") & ")"
734+ of kMap:
735+ var parts: seq[string] = @[]
736+ for (k, v) in f.pairs:
737+ let ke = tryExpr(k, env, c)
738+ if ke.len == 0: return ""
739+ let ve = tryExpr(v, env, c)
740+ if ve.len == 0: return ""
741+ parts.add "(" & ke & ", " & ve & ")"
742+ "mkMap(" & (if parts.len == 0: "newSeq[(Value, Value)]()"
743+ else: "@[" & parts.join(", ") & "]") & ")"
744+ of kList:
745+ if f.items.len == 0: return "mkList(newSeq[Value]())"
746+ let head = f.items[0]
747+ let args = f.items[1 .. ^1]
748+ if head.kind == kSymbol and specialHeads.contains(head.s): return ""
749+ var ids: seq[string] = @[]
750+ if not tryExprs(args, env, c, ids): return ""
751+ if head.kind == kSymbol:
752+ let (iprc, iboxes) = env.lookupIntFn(head.s, args.len)
753+ if iprc.len > 0:
754+ var iids: seq[string] = @[]
755+ var ok = true
756+ for a in args:
757+ let e = intExpr(a, env, c)
758+ if e.len == 0: ok = false; break
759+ iids.add e
760+ if ok:
761+ let callI = iprc & "(" & iids.join(", ") & ")"
762+ return (if iboxes: callI else: "mkInt(" & callI & ")")
763+ let d = lookupDirect(env, head.s, args.len)
764+ if d.prc.len > 0 and (d.cell.len == 0 or c.fnStable(head.s)):
765+ # a self-call, or a name only one def form ever targets
766+ return d.prc & "(" & ids.join(", ") & ")"
767+ if d.prc.len == 0:
768+ let key = head.s & "/" & $args.len
769+ if intrinsics.hasKey(key) and env.lookup(head.s).len == 0:
770+ if c.primStable(head.s):
771+ return intrinsics[key] & "(" & ids.join(", ") & ")"
772+ return intrinsics[key] & "g(" & c.cellFor(head.s) & ", " &
773+ c.coreFor(head.s) & ", " & ids.join(", ") & ")"
774+ let hv = tryExpr(head, env, c)
775+ if hv.len == 0: return ""
776+ "call(" & hv & ", " &
777+ (if ids.len == 0: "emptyArgs" else: "@[" & ids.join(", ") & "]") & ")"
778+ of kFn, kCons, kLazy:
779+ ""
780+
304 proc genInto(f: Value, dst: string, env: Env, c: Ctx) =781 proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
305 if f.isNil:782 if f.isNil:
306 c.line(dst & " = NilV"); return783 c.line(dst & " = NilV"); return
784+ let e = tryExpr(f, env, c)
785+ if e.len > 0:
786+ c.line(dst & " = " & e); return
307 case f.kind787 case f.kind
308 of kNil, kBool, kInt, kFloat, kStr, kKeyword:788 of kNil, kBool, kInt, kFloat, kStr, kKeyword:
309 c.line(dst & " = " & quoteLit(f))789 c.line(dst & " = " & quoteLit(f))
@@ -313,19 +793,19 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
313 else: c.line(dst & " = cellGet(" & c.cellFor(f.s) & ")")793 else: c.line(dst & " = cellGet(" & c.cellFor(f.s) & ")")
314 of kVector:794 of kVector:
315 var ids: seq[string] = @[]795 var ids: seq[string] = @[]
316- for x in f.items: ids.add genExpr(x, env, c)796+ for x in f.items: ids.add genExprTemp(x, env, c)
317 c.line(dst & " = mkVector(" &797 c.line(dst & " = mkVector(" &
318 (if ids.len == 0: "newSeq[Value]()" else: "@[" & ids.join(", ") & "]") & ")")798 (if ids.len == 0: "newSeq[Value]()" else: "@[" & ids.join(", ") & "]") & ")")
319 of kSet:799 of kSet:
320 var ids: seq[string] = @[]800 var ids: seq[string] = @[]
321- for x in f.items: ids.add genExpr(x, env, c)801+ for x in f.items: ids.add genExprTemp(x, env, c)
322 c.line(dst & " = mkSet(" &802 c.line(dst & " = mkSet(" &
323 (if ids.len == 0: "newSeq[Value]()" else: "@[" & ids.join(", ") & "]") & ")")803 (if ids.len == 0: "newSeq[Value]()" else: "@[" & ids.join(", ") & "]") & ")")
324 of kMap:804 of kMap:
325 var parts: seq[string] = @[]805 var parts: seq[string] = @[]
326 for (k, v) in f.pairs:806 for (k, v) in f.pairs:
327- let ki = genExpr(k, env, c)807+ let ki = genExprTemp(k, env, c)
328- let vi = genExpr(v, env, c)808+ let vi = genExprTemp(v, env, c)
329 parts.add "(" & ki & ", " & vi & ")"809 parts.add "(" & ki & ", " & vi & ")"
330 c.line(dst & " = mkMap(" &810 c.line(dst & " = mkMap(" &
331 (if parts.len == 0: "newSeq[(Value, Value)]()" else: "@[" & parts.join(", ") & "]") & ")")811 (if parts.len == 0: "newSeq[(Value, Value)]()" else: "@[" & parts.join(", ") & "]") & ")")
@@ -345,8 +825,7 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
345 return825 return
346 of "if":826 of "if":
347 if args.len < 2: err("Too few arguments to if")827 if args.len < 2: err("Too few arguments to if")
348- let cv = genExpr(args[0], env, c)828+ c.line("if " & genCond(args[0], env, c) & ":")
349- c.line("if truthy(" & cv & "):")
350 c.push; genInto(args[1], dst, env, c); c.pop829 c.push; genInto(args[1], dst, env, c); c.pop
351 c.line("else:")830 c.line("else:")
352 c.push831 c.push
@@ -371,8 +850,16 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
371 if targets.len != args.len:850 if targets.len != args.len:
372 err("Mismatched argument count to recur: expected " & $targets.len &851 err("Mismatched argument count to recur: expected " & $targets.len &
373 ", got " & $args.len)852 ", got " & $args.len)
853+ let targetInts = c.recurInts[^1]
374 var tmps: seq[string] = @[]854 var tmps: seq[string] = @[]
375- for a in args: tmps.add genExpr(a, env, c)855+ for i, a in args:
856+ if targetInts[i]:
857+ let e = intExpr(a, env, c)
858+ let t = c.gensym("t")
859+ c.line("let " & t & ": int64 = " & e)
860+ tmps.add t
861+ else:
862+ tmps.add genExprTemp(a, env, c)
376 for i, t in tmps: c.line(targets[i] & " = " & t)863 for i, t in tmps: c.line(targets[i] & " = " & t)
377 c.line("continue")864 c.line("continue")
378 return865 return
@@ -400,7 +887,7 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
400 if rest.len > 0 and rest[0].kind == kMap: rest = rest[1 .. ^1] # attr map887 if rest.len > 0 and rest[0].kind == kMap: rest = rest[1 .. ^1] # attr map
401 let fv = c.gensym("fn")888 let fv = c.gensym("fn")
402 c.line("var " & fv & ": Value = NilV")889 c.line("var " & fv & ": Value = NilV")
403- genFnForm(rest, env, c, fv, nm)890+ genFnForm(rest, env, c, fv, nm, env, c.cellFor(nm))
404 c.line(dst & " = setVar(" & nimStr(nm) & ", " & fv & ")")891 c.line(dst & " = setVar(" & nimStr(nm) & ", " & fv & ")")
405 return892 return
406 of "defmacro":893 of "defmacro":
@@ -429,22 +916,19 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
429 return916 return
430 of "when":917 of "when":
431 if args.len == 0: err("when requires a test")918 if args.len == 0: err("when requires a test")
432- let cv = genExpr(args[0], env, c)919+ c.line("if " & genCond(args[0], env, c) & ":")
433- c.line("if truthy(" & cv & "):")
434 c.push; genBody(args[1 .. ^1], dst, env, c); c.pop920 c.push; genBody(args[1 .. ^1], dst, env, c); c.pop
435 c.line("else:")921 c.line("else:")
436 c.push; c.line(dst & " = NilV"); c.pop922 c.push; c.line(dst & " = NilV"); c.pop
437 return923 return
438 of "when-not":924 of "when-not":
439- let cv = genExpr(args[0], env, c)925+ c.line("if not (" & genCond(args[0], env, c) & "):")
440- c.line("if not truthy(" & cv & "):")
441 c.push; genBody(args[1 .. ^1], dst, env, c); c.pop926 c.push; genBody(args[1 .. ^1], dst, env, c); c.pop
442 c.line("else:")927 c.line("else:")
443 c.push; c.line(dst & " = NilV"); c.pop928 c.push; c.line(dst & " = NilV"); c.pop
444 return929 return
445 of "if-not":930 of "if-not":
446- let cv = genExpr(args[0], env, c)931+ c.line("if not (" & genCond(args[0], env, c) & "):")
447- c.line("if not truthy(" & cv & "):")
448 c.push; genInto(args[1], dst, env, c); c.pop932 c.push; genInto(args[1], dst, env, c); c.pop
449 c.line("else:")933 c.line("else:")
450 c.push934 c.push
@@ -460,8 +944,7 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
460 if isSym(args[i], "else") or (args[i].kind == kKeyword and args[i].s == "else"):944 if isSym(args[i], "else") or (args[i].kind == kKeyword and args[i].s == "else"):
461 genInto(args[i + 1], dst, env, c)945 genInto(args[i + 1], dst, env, c)
462 break946 break
463- let cv = genExpr(args[i], env, c)947+ c.line("if " & genCond(args[i], env, c) & ":")
464- c.line("if truthy(" & cv & "):")
465 c.push948 c.push
466 genInto(args[i + 1], dst, env, c)949 genInto(args[i + 1], dst, env, c)
467 c.pop950 c.pop
@@ -474,12 +957,11 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
474 let b = args[0]957 let b = args[0]
475 if b.kind != kVector or b.items.len != 2: err(head.s & " requires [sym test]")958 if b.kind != kVector or b.items.len != 2: err(head.s & " requires [sym test]")
476 let nm = symName(b.items[0])959 let nm = symName(b.items[0])
477- let tv = genExpr(b.items[1], env, c)960+ let id = c.gensym("l" & mangle(nm))
478- c.line("if truthy(" & tv & "):")961+ c.line("var " & id & ": Value = " & genExpr(b.items[1], env, c))
962+ c.line("if truthy(" & id & "):")
479 c.push963 c.push
480 let benv = newEnv(env)964 let benv = newEnv(env)
481- let id = c.gensym("l" & mangle(nm))
482- c.line("var " & id & ": Value = " & tv)
483 benv.locals[nm] = id965 benv.locals[nm] = id
484 if head.s == "when-let": genBody(args[1 .. ^1], dst, benv, c)966 if head.s == "when-let": genBody(args[1 .. ^1], dst, benv, c)
485 else: genInto(args[1], dst, benv, c)967 else: genInto(args[1], dst, benv, c)
@@ -595,10 +1077,26 @@ import runtime, core
595 proc cljMain() =1077 proc cljMain() =
596 """1078 """
597 1079
1080+proc collectDefs(f: Value, into: var CountTable[string]) =
1081+ ## Every name this program can rebind at runtime. `def` and `defn` are the
1082+ ## only paths to setVar, and clonim has no eval, no defmacro and no intern,
1083+ ## so a name that no def form targets holds whatever registerCore gave it for
1084+ ## the life of the process. That is what lets call sites drop the cell guard
1085+ ## and lets the analyzer trust `+` to be arithmetic.
1086+ if f.isNil or f.kind != kList or f.items.len == 0: return
1087+ let head = f.items[0]
1088+ if head.kind == kSymbol and head.s in ["def", "defn", "defn-"] and
1089+ f.items.len > 1 and f.items[1].kind == kSymbol:
1090+ into.inc f.items[1].s
1091+ for x in f.items: collectDefs(x, into)
1092+
598 proc compileForms*(forms: seq[Value]): string =1093 proc compileForms*(forms: seq[Value]): string =
599 let c = Ctx(body: @[], indent: 1, counter: 0, recurStack: @[],1094 let c = Ctx(body: @[], indent: 1, counter: 0, recurStack: @[],
1095+ recurInts: @[], cores: initTable[string, string](),
600 defined: initHashSet[string](), prelude: @[],1096 defined: initHashSet[string](), prelude: @[],
601- cells: initTable[string, string]())1097+ cells: initTable[string, string](),
1098+ defCounts: initCountTable[string]())
1099+ for f in forms: collectDefs(f, c.defCounts)
602 let env = newEnv()1100 let env = newEnv()
603 for f in forms:1101 for f in forms:
604 let t = c.gensym("top")1102 let t = c.gensym("top")
modified src/core.nim +98 -3
@@ -40,6 +40,101 @@ proc cmpChain(args: seq[Value], ok: proc (c: int): bool): Value =
4040 if not ok(c): return FalseV
4141 TrueV
4242
43+# ------------------------------------------------------- inlinable primitives
44+## Two-argument forms of the arithmetic and comparison builtins, exported so
45+## call sites can inline them instead of dispatching through a closure and an
46+## argument list, and so the analyzer has something to compile an integer
47+## expression down to. Each takes the int/int path in a couple of instructions
48+## and otherwise falls back to the same numeric-tower behaviour as the generic
49+## builtin.
50+
51+## Integer division that reports rather than trapping: Nim's `div` raises an
52+## uncatchable defect on a zero divisor, where `/` here already errored.
53+proc idiv*(a, b: int64): int64 {.inline.} =
54+ if b == 0: err("Divide by zero")
55+ a div b
56+
57+proc irem*(a, b: int64): int64 {.inline.} =
58+ if b == 0: err("Divide by zero")
59+ a mod b
60+
61+proc add2*(a, b: Value): Value {.inline.} =
62+ if a.kind == kInt and b.kind == kInt: mkInt(a.i + b.i)
63+ else: mkFloat(num(a) + num(b))
64+
65+proc sub2*(a, b: Value): Value {.inline.} =
66+ if a.kind == kInt and b.kind == kInt: mkInt(a.i - b.i)
67+ else: mkFloat(num(a) - num(b))
68+
69+proc mul2*(a, b: Value): Value {.inline.} =
70+ if a.kind == kInt and b.kind == kInt: mkInt(a.i * b.i)
71+ else: mkFloat(num(a) * num(b))
72+
73+proc lt2*(a, b: Value): Value {.inline.} =
74+ if a.kind == kInt and b.kind == kInt: mkBool(a.i < b.i)
75+ else: mkBool(num(a) < num(b))
76+
77+proc gt2*(a, b: Value): Value {.inline.} =
78+ if a.kind == kInt and b.kind == kInt: mkBool(a.i > b.i)
79+ else: mkBool(num(a) > num(b))
80+
81+proc le2*(a, b: Value): Value {.inline.} =
82+ if a.kind == kInt and b.kind == kInt: mkBool(a.i <= b.i)
83+ else: mkBool(num(a) <= num(b))
84+
85+proc ge2*(a, b: Value): Value {.inline.} =
86+ if a.kind == kInt and b.kind == kInt: mkBool(a.i >= b.i)
87+ else: mkBool(num(a) >= num(b))
88+
89+proc eq2*(a, b: Value): Value {.inline.} =
90+ if a.kind == kInt and b.kind == kInt: mkBool(a.i == b.i)
91+ else: mkBool(equals(a, b))
92+
93+proc ne2*(a, b: Value): Value {.inline.} =
94+ if a.kind == kInt and b.kind == kInt: mkBool(a.i != b.i)
95+ else: mkBool(not equals(a, b))
96+
97+proc inc1*(a: Value): Value {.inline.} =
98+ if a.kind == kInt: mkInt(a.i + 1) else: mkFloat(num(a) + 1.0)
99+
100+proc dec1*(a: Value): Value {.inline.} =
101+ if a.kind == kInt: mkInt(a.i - 1) else: mkFloat(num(a) - 1.0)
102+
103+## Guarded forms: the whole call site, cell check included, as one expression,
104+## for the call sites where a def in the program can still rebind the name.
105+proc add2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
106+ if cellIs(c, k): add2(a, b) else: call(cellGet(c), @[a, b])
107+
108+proc sub2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
109+ if cellIs(c, k): sub2(a, b) else: call(cellGet(c), @[a, b])
110+
111+proc mul2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
112+ if cellIs(c, k): mul2(a, b) else: call(cellGet(c), @[a, b])
113+
114+proc lt2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
115+ if cellIs(c, k): lt2(a, b) else: call(cellGet(c), @[a, b])
116+
117+proc gt2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
118+ if cellIs(c, k): gt2(a, b) else: call(cellGet(c), @[a, b])
119+
120+proc le2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
121+ if cellIs(c, k): le2(a, b) else: call(cellGet(c), @[a, b])
122+
123+proc ge2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
124+ if cellIs(c, k): ge2(a, b) else: call(cellGet(c), @[a, b])
125+
126+proc eq2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
127+ if cellIs(c, k): eq2(a, b) else: call(cellGet(c), @[a, b])
128+
129+proc ne2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
130+ if cellIs(c, k): ne2(a, b) else: call(cellGet(c), @[a, b])
131+
132+proc inc1g*(c: VarCell, k: Value, a: Value): Value {.inline.} =
133+ if cellIs(c, k): inc1(a) else: call(cellGet(c), @[a])
134+
135+proc dec1g*(c: VarCell, k: Value, a: Value): Value {.inline.} =
136+ if cellIs(c, k): dec1(a) else: call(cellGet(c), @[a])
137+
43138 proc getIn(coll, k, dflt: Value): Value =
44139 if coll.isNil or coll.kind == kNil: return dflt
45140 case coll.kind
@@ -232,11 +327,11 @@ proc registerCore*() =
232327 for i in 1 ..< a.len:
233328 if a[i].kind == kInt and a[i].i == 0: err("Divide by zero")
234329 arith("/", a, 1, proc (x, y: int64): int64 = x div y, proc (x, y: float64): float64 = x / y)
235- def "quot", proc (a: seq[Value]): Value = mkInt(intOf(a[0]) div intOf(a[1]))
236- def "rem", proc (a: seq[Value]): Value = mkInt(intOf(a[0]) mod intOf(a[1]))
330+ def "quot", proc (a: seq[Value]): Value = mkInt(idiv(intOf(a[0]), intOf(a[1])))
331+ def "rem", proc (a: seq[Value]): Value = mkInt(irem(intOf(a[0]), intOf(a[1])))
237332 def "mod", proc (a: seq[Value]): Value =
238333 let x = intOf(a[0]); let y = intOf(a[1])
239- var r = x mod y
334+ var r = irem(x, y)
240335 if r != 0 and ((r < 0) != (y < 0)): r += y
241336 mkInt(r)
242337 def "inc", proc (a: seq[Value]): Value =
@@ -40,6 +40,101 @@ proc cmpChain(args: seq[Value], ok: proc (c: int): bool): Value =
40 if not ok(c): return FalseV40 if not ok(c): return FalseV
41 TrueV41 TrueV
42 42
43+# ------------------------------------------------------- inlinable primitives
44+## Two-argument forms of the arithmetic and comparison builtins, exported so
45+## call sites can inline them instead of dispatching through a closure and an
46+## argument list, and so the analyzer has something to compile an integer
47+## expression down to. Each takes the int/int path in a couple of instructions
48+## and otherwise falls back to the same numeric-tower behaviour as the generic
49+## builtin.
50+
51+## Integer division that reports rather than trapping: Nim's `div` raises an
52+## uncatchable defect on a zero divisor, where `/` here already errored.
53+proc idiv*(a, b: int64): int64 {.inline.} =
54+ if b == 0: err("Divide by zero")
55+ a div b
56+
57+proc irem*(a, b: int64): int64 {.inline.} =
58+ if b == 0: err("Divide by zero")
59+ a mod b
60+
61+proc add2*(a, b: Value): Value {.inline.} =
62+ if a.kind == kInt and b.kind == kInt: mkInt(a.i + b.i)
63+ else: mkFloat(num(a) + num(b))
64+
65+proc sub2*(a, b: Value): Value {.inline.} =
66+ if a.kind == kInt and b.kind == kInt: mkInt(a.i - b.i)
67+ else: mkFloat(num(a) - num(b))
68+
69+proc mul2*(a, b: Value): Value {.inline.} =
70+ if a.kind == kInt and b.kind == kInt: mkInt(a.i * b.i)
71+ else: mkFloat(num(a) * num(b))
72+
73+proc lt2*(a, b: Value): Value {.inline.} =
74+ if a.kind == kInt and b.kind == kInt: mkBool(a.i < b.i)
75+ else: mkBool(num(a) < num(b))
76+
77+proc gt2*(a, b: Value): Value {.inline.} =
78+ if a.kind == kInt and b.kind == kInt: mkBool(a.i > b.i)
79+ else: mkBool(num(a) > num(b))
80+
81+proc le2*(a, b: Value): Value {.inline.} =
82+ if a.kind == kInt and b.kind == kInt: mkBool(a.i <= b.i)
83+ else: mkBool(num(a) <= num(b))
84+
85+proc ge2*(a, b: Value): Value {.inline.} =
86+ if a.kind == kInt and b.kind == kInt: mkBool(a.i >= b.i)
87+ else: mkBool(num(a) >= num(b))
88+
89+proc eq2*(a, b: Value): Value {.inline.} =
90+ if a.kind == kInt and b.kind == kInt: mkBool(a.i == b.i)
91+ else: mkBool(equals(a, b))
92+
93+proc ne2*(a, b: Value): Value {.inline.} =
94+ if a.kind == kInt and b.kind == kInt: mkBool(a.i != b.i)
95+ else: mkBool(not equals(a, b))
96+
97+proc inc1*(a: Value): Value {.inline.} =
98+ if a.kind == kInt: mkInt(a.i + 1) else: mkFloat(num(a) + 1.0)
99+
100+proc dec1*(a: Value): Value {.inline.} =
101+ if a.kind == kInt: mkInt(a.i - 1) else: mkFloat(num(a) - 1.0)
102+
103+## Guarded forms: the whole call site, cell check included, as one expression,
104+## for the call sites where a def in the program can still rebind the name.
105+proc add2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
106+ if cellIs(c, k): add2(a, b) else: call(cellGet(c), @[a, b])
107+
108+proc sub2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
109+ if cellIs(c, k): sub2(a, b) else: call(cellGet(c), @[a, b])
110+
111+proc mul2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
112+ if cellIs(c, k): mul2(a, b) else: call(cellGet(c), @[a, b])
113+
114+proc lt2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
115+ if cellIs(c, k): lt2(a, b) else: call(cellGet(c), @[a, b])
116+
117+proc gt2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
118+ if cellIs(c, k): gt2(a, b) else: call(cellGet(c), @[a, b])
119+
120+proc le2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
121+ if cellIs(c, k): le2(a, b) else: call(cellGet(c), @[a, b])
122+
123+proc ge2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
124+ if cellIs(c, k): ge2(a, b) else: call(cellGet(c), @[a, b])
125+
126+proc eq2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
127+ if cellIs(c, k): eq2(a, b) else: call(cellGet(c), @[a, b])
128+
129+proc ne2g*(c: VarCell, k: Value, a, b: Value): Value {.inline.} =
130+ if cellIs(c, k): ne2(a, b) else: call(cellGet(c), @[a, b])
131+
132+proc inc1g*(c: VarCell, k: Value, a: Value): Value {.inline.} =
133+ if cellIs(c, k): inc1(a) else: call(cellGet(c), @[a])
134+
135+proc dec1g*(c: VarCell, k: Value, a: Value): Value {.inline.} =
136+ if cellIs(c, k): dec1(a) else: call(cellGet(c), @[a])
137+
43 proc getIn(coll, k, dflt: Value): Value =138 proc getIn(coll, k, dflt: Value): Value =
44 if coll.isNil or coll.kind == kNil: return dflt139 if coll.isNil or coll.kind == kNil: return dflt
45 case coll.kind140 case coll.kind
@@ -232,11 +327,11 @@ proc registerCore*() =
232 for i in 1 ..< a.len:327 for i in 1 ..< a.len:
233 if a[i].kind == kInt and a[i].i == 0: err("Divide by zero")328 if a[i].kind == kInt and a[i].i == 0: err("Divide by zero")
234 arith("/", a, 1, proc (x, y: int64): int64 = x div y, proc (x, y: float64): float64 = x / y)329 arith("/", a, 1, proc (x, y: int64): int64 = x div y, proc (x, y: float64): float64 = x / y)
235- def "quot", proc (a: seq[Value]): Value = mkInt(intOf(a[0]) div intOf(a[1]))330+ def "quot", proc (a: seq[Value]): Value = mkInt(idiv(intOf(a[0]), intOf(a[1])))
236- def "rem", proc (a: seq[Value]): Value = mkInt(intOf(a[0]) mod intOf(a[1]))331+ def "rem", proc (a: seq[Value]): Value = mkInt(irem(intOf(a[0]), intOf(a[1])))
237 def "mod", proc (a: seq[Value]): Value =332 def "mod", proc (a: seq[Value]): Value =
238 let x = intOf(a[0]); let y = intOf(a[1])333 let x = intOf(a[0]); let y = intOf(a[1])
239- var r = x mod y334+ var r = irem(x, y)
240 if r != 0 and ((r < 0) != (y < 0)): r += y335 if r != 0 and ((r < 0) != (y < 0)): r += y
241 mkInt(r)336 mkInt(r)
242 def "inc", proc (a: seq[Value]): Value =337 def "inc", proc (a: seq[Value]): Value =
modified src/runtime.nim +16 -4
@@ -442,10 +442,16 @@ proc cursor*(v: Value): Cursor =
442442 else: Cursor(isNode: false, backing: toSeq(v))
443443
444444 proc hasNext*(c: var Cursor): bool =
445- if c.isNode:
446- c.node = force(c.node)
447- not c.node.isNil and c.node.kind == kCons
448- else: c.idx < c.backing.len
445+ if not c.isNode: return c.idx < c.backing.len
446+ c.node = force(c.node)
447+ if c.node.isNil or c.node.kind == kNil: return false
448+ if c.node.kind == kCons: return true
449+ # The tail bottomed out in a concrete collection, as `(cons x [1 2])` does.
450+ # Switch to indexing it rather than reporting the seq as finished.
451+ c.isNode = false
452+ c.backing = toSeq(c.node)
453+ c.idx = 0
454+ c.idx < c.backing.len
449455
450456 proc next*(c: var Cursor): Value =
451457 if c.isNode:
@@ -732,3 +738,9 @@ proc toSeq*(v: Value): seq[Value] =
732738 else: err("Don't know how to create seq from: " & prStr(v))
733739
734740 let emptyArgs*: seq[Value] = @[]
741+
742+## True while a var still holds the exact fn a call site was compiled against.
743+## Call sites that bind a known-arity fn or an inlined primitive directly guard
744+## on this, so a later `def` that rebinds the name still takes effect.
745+proc cellIs*(c: VarCell, v: Value): bool {.inline.} =
746+ c.bound and c.v == v
@@ -442,10 +442,16 @@ proc cursor*(v: Value): Cursor =
442 else: Cursor(isNode: false, backing: toSeq(v))442 else: Cursor(isNode: false, backing: toSeq(v))
443 443
444 proc hasNext*(c: var Cursor): bool =444 proc hasNext*(c: var Cursor): bool =
445- if c.isNode:445+ if not c.isNode: return c.idx < c.backing.len
446- c.node = force(c.node)446+ c.node = force(c.node)
447- not c.node.isNil and c.node.kind == kCons447+ if c.node.isNil or c.node.kind == kNil: return false
448- else: c.idx < c.backing.len448+ if c.node.kind == kCons: return true
449+ # The tail bottomed out in a concrete collection, as `(cons x [1 2])` does.
450+ # Switch to indexing it rather than reporting the seq as finished.
451+ c.isNode = false
452+ c.backing = toSeq(c.node)
453+ c.idx = 0
454+ c.idx < c.backing.len
449 455
450 proc next*(c: var Cursor): Value =456 proc next*(c: var Cursor): Value =
451 if c.isNode:457 if c.isNode:
@@ -732,3 +738,9 @@ proc toSeq*(v: Value): seq[Value] =
732 else: err("Don't know how to create seq from: " & prStr(v))738 else: err("Don't know how to create seq from: " & prStr(v))
733 739
734 let emptyArgs*: seq[Value] = @[]740 let emptyArgs*: seq[Value] = @[]
741+
742+## True while a var still holds the exact fn a call site was compiled against.
743+## Call sites that bind a known-arity fn or an inlined primitive directly guard
744+## on this, so a later `def` that rebinds the name still takes effect.
745+proc cellIs*(c: VarCell, v: Value): bool {.inline.} =
746+ c.bound and c.v == v
added tests/directcalls.expected +7 -0
new file mode 100644
@@ -0,0 +1,7 @@
1+10
2+105
3+4 7 10
4+(2 3 4)
5+50
6+3628800
7+true true
new file mode 100644
@@ -0,0 +1,7 @@
1+10
2+105
3+4 7 10
4+(2 3 4)
5+50
6+3628800
7+true true
added tests/evalorder.expected +11 -0
new file mode 100644
@@ -0,0 +1,11 @@
1+123 [1 2 3]
2+:done [:a :b]
3+123 [1 2 3]
4+10
5+[2 1]
6+7 [7]
7+8 [8]
8+false :first []
9+[1 2] [1 2]
10+1 [:k1 :v1]
11+[12 2]
new file mode 100644
@@ -0,0 +1,11 @@
1+123 [1 2 3]
2+:done [:a :b]
3+123 [1 2 3]
4+10
5+[2 1]
6+7 [7]
7+8 [8]
8+false :first []
9+[1 2] [1 2]
10+1 [:k1 :v1]
11+[12 2]
added tests/forms.expected +7 -0
new file mode 100644
@@ -0,0 +1,7 @@
1+dotimes 10
2+doseq 19
3+caught: Divide by zero
4+finally ran: 99
5+12 9
6+:b
7+nested 7
new file mode 100644
@@ -0,0 +1,7 @@
1+dotimes 10
2+doseq 19
3+caught: Divide by zero
4+finally ran: 99
5+12 9
6+:b
7+nested 7
added tests/intrinsics.expected +10 -0
new file mode 100644
@@ -0,0 +1,10 @@
1+3 2 20 8 6
2+3.5 1.0 0.75 2.5
3+true false true false
4+true true
5+true false true true
6+true true false
7+6 true true
8+12
9+10
10+999
new file mode 100644
@@ -0,0 +1,10 @@
1+3 2 20 8 6
2+3.5 1.0 0.75 2.5
3+true false true false
4+true true
5+true false true true
6+true true false
7+6 true true
8+12
9+10
10+999
added tests/rebinding.expected +5 -0
new file mode 100644
@@ -0,0 +1,5 @@
1+11
2+1000
3+10
4+0
5+12
new file mode 100644
@@ -0,0 +1,5 @@
1+11
2+1000
3+10
4+0
5+12
added tests/seqlib.expected +9 -0
new file mode 100644
@@ -0,0 +1,9 @@
1+(0 1 2 3 4) (2 3 4) (0 3 6 9) (5 3 1)
2+() () () ()
3+3 2 4 2 0
4+true false true true
5+(1 2)
6+(:one :two)
7+0 7 106 6
8+(2 3) 5 ()
9+3 3 3 3
new file mode 100644
@@ -0,0 +1,9 @@
1+(0 1 2 3 4) (2 3 4) (0 3 6 9) (5 3 1)
2+() () () ()
3+3 2 4 2 0
4+true false true true
5+(1 2)
6+(:one :two)
7+0 7 106 6
8+(2 3) 5 ()
9+3 3 3 3
added tests/typespec.expected +15 -0
new file mode 100644
@@ -0,0 +1,15 @@
1+1.5
2+aaa
3+[0 1 2]
4+1.5
5+[0 1 2]
6+42 3.0
7+12 ab
8+(2 4 6) 6
9+25
10+3 -3 1 -1 6 4
11+true true true true
12+caught
13+aaa
14+5050
15+4500001500000
new file mode 100644
@@ -0,0 +1,15 @@
1+1.5
2+aaa
3+[0 1 2]
4+1.5
5+[0 1 2]
6+42 3.0
7+12 ab
8+(2 4 6) 6
9+25
10+3 -3 1 -1 6 4
11+true true true true
12+caught
13+aaa
14+5050
15+4500001500000