nandi/clonim · 4de7343

nandi/clonimpublic⑂ Fork 0

⑂ 4de7343

Lazy seqs for map, filter, range and the rest of the seq library

map/filter/remove/range/take/drop/take-while/drop-while/concat/
map-indexed now return lazy seqs, and iterate/repeat/repeatedly/cycle/
doall/dorun join them. An element is computed on first demand and
memoized, so infinite seqs are ordinary values and a consumer that stops
early never pays for the rest:

  (take 5 (filter even? (range)))                  => (0 2 4 6 8)
  (first (filter odd? (map inc (range 2000000))))  671ms -> 0ms

Two new Value kinds: kCons (head + tail) and kLazy (a memoizing thunk).
Producers all share one shape — capture a Cursor, return a thunk that
advances a copy of it and yields one cons — so no producer recurses per
element and nothing is computed at construction time.

Cursor is the single way core walks a collection: it follows a cons/lazy
chain link by link and indexes concrete collections directly, so no
builtin materializes more of a seq than it was asked for. nth, first,
rest, seq, empty? and `& rest` destructuring stop at the element they
need; count, reduce and printing realize the whole seq. cons is now O(1)
rather than a copy.

Teardown needed a hand-written =destroy. ARC frees a linked structure by
recursing into it, so dropping a million-element seq meant a million
destructor frames and a segfault. Value now hands a cons or lazy tail to
a worklist and drains it in a loop; nothing shared is mutated, so a tail
another seq still holds survives untouched. Verified with a 1M-element
teardown and a 40-round churn of lazy seqs, vectors, maps, sets, strings
and closures: flat RSS, clean exit. --mm:refc would also have avoided the
recursion but cost 4.5x on map/set writes.

The trade is the usual one: a fully realized lazy seq allocates a cons
and a thunk per element where the eager path filled one flat seq, so
realizing all of (map inc (range 1000000)) went 290ms -> 570ms. Clojure
buys that back with 32-element chunking; not chunking is why take 3
computes exactly three elements here. fib(30) is unchanged.

Also fixes `& rest` destructuring, which realized the whole collection
and so hung on an infinite seq.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

nandithebull committed 2026-09-20T16:30:33-07:00 Browse files

4de7343 parent: 86373e5

modified README.md +39 -4 Viewed

@@ -66,6 +66,37 @@ Clojure's small array-maps are, without giving up hashed lookup.
66	66	The set column is the honest shape of the old representation: `conj` rebuilt the
67	67	whole set and re-scanned it for duplicates, so building one was O(n³).
68	68
	69	+Seqs are lazy. `map`, `filter`, `remove`, `range`, `take`, `drop`,
	70	+`take-while`, `drop-while`, `concat`, `map-indexed`, `iterate`, `repeat`,
	71	+`repeatedly` and `cycle` return a chain of thunks: each element is computed on
	72	+first demand and memoized, so infinite seqs are ordinary values and a consumer
	73	+that stops early never pays for the rest.
	74	+
	75	+```clojure
	76	+(take 5 (filter even? (range))) ;=> (0 2 4 6 8)
	77	+(first (filter odd? (map inc (range 2000000)))) ; 671 ms eager -> 0 ms lazy
	78	+```
	79	+
	80	+Everything in `core` walks collections through one `Cursor`, which follows a
	81	+cons/lazy chain link by link and indexes concrete collections directly, so a
	82	+builtin never materializes more of a seq than it was asked for. `nth`, `first`,
	83	+`rest`, `seq`, `empty?` and `& rest` destructuring all stop at the element they
	84	+need; `count`, `reduce` and printing realize the whole seq, which is what those
	85	+mean.
	86	+
	87	+The cost is the usual one: a fully realized lazy seq allocates a cons cell and
	88	+a thunk per element, where the eager version filled one flat `seq`. Realizing
	89	+all of `(map inc (range 1000000))` went from 290 ms to 570 ms. Clojure buys most
	90	+of that back by realizing in 32-element chunks; clonim does not chunk yet, which
	91	+is why its laziness is exact — `take 3` computes exactly three elements, not
	92	+thirty-two.
	93	+
	94	+Long chains need one piece of care. ARC frees a linked structure by recursing
	95	+into it, so dropping a million-element seq means a million destructor frames and
	96	+a segfault. `Value` therefore has a hand-written `=destroy` that hands a cons or
	97	+lazy tail to a worklist and drains it in a loop. Nothing shared is mutated, so a
	98	+tail another seq still holds simply survives.
	99	+
69	100	## What works
70	101
71	102	`def` `defn` (multi-arity, varargs, docstrings) `fn` (named, self-recursive)
@@ -78,15 +109,19 @@ Data: nil, bool, int, float, string, keyword, symbol, list, vector, map, set —
78	109	persistent, with structural equality, hashing, and Clojure-shaped printing. Atoms, closures, `comp`,
79	110	`partial`, `juxt`, the usual seq library, `clojure.string/*`.
80	111
	112	+Lazy seqs: `iterate` `repeat` `repeatedly` `cycle` `doall` `dorun`, and the seq
	113	+library above returns them where Clojure does.
	114	+
81	115	## What doesn't (yet)
82	116
83	117	- `defmacro`. The macro set is fixed and expanded by the compiler. User
84	118	macros need the compiler to be able to evaluate code at compile time —
85	119	the honest fix is to bootstrap clonim in itself, or embed an interpreter.
86		-- Laziness. `map`/`filter`/`range` are eager. Infinite seqs will hang.
87		-- Laziness for the seq library. Most of `core` still materialises a
88		- `seq[Value]` on the way in and out, so even with persistent vectors, `map`
89		- over a big collection allocates twice.
	120	+- Chunked seqs. Lazy seqs are unchunked, so full realization allocates two
	121	+ cells per element and runs ~2× slower than the old eager path. 32-element
	122	+ chunking is the fix, at the cost of exact demand.
	123	+- Destructuring in parameter lists. `(let [[a b] xs] …)` works; `(defn f
	124	+ [[a b]] …)` does not.
90	125	- Protocols/records, namespaces (`ns` is parsed and ignored), refs/agents,
91	126	`#()` literals, syntax-quote, transducers, Nim interop.
92	127

  The set column is the honest shape of the old representation: `conj` rebuilt the
  whole set and re-scanned it for duplicates, so building one was O(n³).
++**Seqs are lazy.** `map`, `filter`, `remove`, `range`, `take`, `drop`,
++`take-while`, `drop-while`, `concat`, `map-indexed`, `iterate`, `repeat`,
++`repeatedly` and `cycle` return a chain of thunks: each element is computed on
++first demand and memoized, so infinite seqs are ordinary values and a consumer
++that stops early never pays for the rest.
++
++```clojure
++(take 5 (filter even? (range)))            ;=> (0 2 4 6 8)
++(first (filter odd? (map inc (range 2000000))))   ; 671 ms eager -> 0 ms lazy
++```
++
++Everything in `core` walks collections through one `Cursor`, which follows a
++cons/lazy chain link by link and indexes concrete collections directly, so a
++builtin never materializes more of a seq than it was asked for. `nth`, `first`,
++`rest`, `seq`, `empty?` and `& rest` destructuring all stop at the element they
++need; `count`, `reduce` and printing realize the whole seq, which is what those
++mean.
++
++The cost is the usual one: a fully realized lazy seq allocates a cons cell and
++a thunk per element, where the eager version filled one flat `seq`. Realizing
++all of `(map inc (range 1000000))` went from 290 ms to 570 ms. Clojure buys most
++of that back by realizing in 32-element chunks; clonim does not chunk yet, which
++is why its laziness is exact — `take 3` computes exactly three elements, not
++thirty-two.
++
++Long chains need one piece of care. ARC frees a linked structure by recursing
++into it, so dropping a million-element seq means a million destructor frames and
++a segfault. `Value` therefore has a hand-written `=destroy` that hands a cons or
++lazy tail to a worklist and drains it in a loop. Nothing shared is mutated, so a
++tail another seq still holds simply survives.
++
  ## What works
  `def` `defn` (multi-arity, varargs, docstrings) `fn` (named, self-recursive)
  persistent, with structural equality, hashing, and Clojure-shaped printing. Atoms, closures, `comp`,
  `partial`, `juxt`, the usual seq library, `clojure.string/*`.
++Lazy seqs: `iterate` `repeat` `repeatedly` `cycle` `doall` `dorun`, and the seq
++library above returns them where Clojure does.
++
  ## What doesn't (yet)
  - **`defmacro`.** The macro set is fixed and expanded by the compiler. User
    macros need the compiler to be able to *evaluate* code at compile time —
    the honest fix is to bootstrap clonim in itself, or embed an interpreter.
--- **Laziness.** `map`/`filter`/`range` are eager. Infinite seqs will hang.
++- **Chunked seqs.** Lazy seqs are unchunked, so full realization allocates two
--- **Laziness for the seq library.** Most of `core` still materialises a
++  cells per element and runs ~2× slower than the old eager path. 32-element
--  `seq[Value]` on the way in and out, so even with persistent vectors, `map`
++  chunking is the fix, at the cost of exact demand.
--  over a big collection allocates twice.
++- **Destructuring in parameter lists.** `(let [[a b] xs] …)` works; `(defn f
++  [[a b]] …)` does not.
  - Protocols/records, namespaces (`ns` is parsed and ignored), refs/agents,
    `#()` literals, syntax-quote, transducers, Nim interop.

added examples/lazy-bench.clj +8 -0 Viewed

new file mode 100644
@@ -0,0 +1,8 @@
	1	+(def n 2000000)
	2	+(let [t0 (now-ms)
	3	+ a (first (filter odd? (map inc (range n))))
	4	+ t1 (now-ms)
	5	+ b (reduce + 0 (take 10 (map (fn [x] (* x x)) (range n))))
	6	+ t2 (now-ms)]
	7	+ (println "first of filter/map over" n ":" (- t1 t0) "ms =" a)
	8	+ (println "sum of take 10 over " n ":" (- t2 t1) "ms =" b))

++(def n 2000000)
++(let [t0 (now-ms)
++      a  (first (filter odd? (map inc (range n))))
++      t1 (now-ms)
++      b  (reduce + 0 (take 10 (map (fn [x] (* x x)) (range n))))
++      t2 (now-ms)]
++  (println "first of filter/map over" n ":" (- t1 t0) "ms  =" a)
++  (println "sum of take 10 over    " n ":" (- t2 t1) "ms  =" b))

added examples/lazy.clj +42 -0 Viewed

new file mode 100644
@@ -0,0 +1,42 @@
	1	+;; Lazy seqs: map/filter/range and friends compute only what is asked for.
	2	+
	3	+;; infinite sources are ordinary values
	4	+(println (take 5 (range)))
	5	+(println (take 5 (map inc (range))))
	6	+(println (take 5 (filter even? (range))))
	7	+(println (take 5 (iterate (fn [x] (* 2 x)) 1)))
	8	+(println (take 4 (repeat :x)) (repeat 3 :y))
	9	+(println (take 7 (cycle [1 2 3])))
	10	+(println (take 3 (drop 100 (range))))
	11	+(println (take-while (fn [x] (< x 5)) (range)))
	12	+(println (take 3 (drop-while (fn [x] (< x 10)) (range))))
	13	+(println (take 5 (concat [1 2] (range))))
	14	+(println (first (map inc (range))) (second (range)) (nth (range) 1000))
	15	+
	16	+;; nothing beyond the demand is computed, and each cell is computed once
	17	+(def calls (atom 0))
	18	+(def xs (map (fn [x] (reset! calls (inc (deref calls))) x) (range 1000)))
	19	+(println "built, calls so far:" (deref calls))
	20	+(def three (doall (take 3 xs)))
	21	+(println "took" three "- calls:" (deref calls))
	22	+(println "took" (doall (take 3 xs)) "- calls:" (deref calls) "(memoized)")
	23	+
	24	+;; composing lazily builds no intermediate collections
	25	+(println (reduce + 0 (take 10 (filter odd? (map (fn [x] (* x x)) (range))))))
	26	+
	27	+;; destructuring walks only as far as the pattern needs
	28	+(let [[a b & more] (range)]
	29	+ (println a b (take 3 more)))
	30	+
	31	+;; finite collections behave exactly as before
	32	+(println (map inc [1 2 3]) (filter even? [1 2 3 4]))
	33	+(println (range 5) (range 2 8 2) (range 5 0 -1))
	34	+(println (count (range 100)) (empty? (range 0)) (seq (range 0)) (seq (range 2)))
	35	+(println (= (range 3) [0 1 2]) (= (map inc [0 1]) (list 1 2)))
	36	+(println (vec (take 3 (range))) (sort (take 4 (map (fn [x] (- 9 x)) (range)))))
	37	+(println (cons 0 (range 3)) (rest (range 3)) (next (range 1)) (last (take 4 (range))))
	38	+(doseq [x (take 3 (map inc (range)))] (print x ""))
	39	+(println)
	40	+
	41	+;; realizing a long seq is iterative: no stack growth, no deep teardown
	42	+(println (count (take 200000 (range))) (nth (iterate inc 0) 200000))

++;; Lazy seqs: map/filter/range and friends compute only what is asked for.
++
++;; infinite sources are ordinary values
++(println (take 5 (range)))
++(println (take 5 (map inc (range))))
++(println (take 5 (filter even? (range))))
++(println (take 5 (iterate (fn [x] (* 2 x)) 1)))
++(println (take 4 (repeat :x)) (repeat 3 :y))
++(println (take 7 (cycle [1 2 3])))
++(println (take 3 (drop 100 (range))))
++(println (take-while (fn [x] (< x 5)) (range)))
++(println (take 3 (drop-while (fn [x] (< x 10)) (range))))
++(println (take 5 (concat [1 2] (range))))
++(println (first (map inc (range))) (second (range)) (nth (range) 1000))
++
++;; nothing beyond the demand is computed, and each cell is computed once
++(def calls (atom 0))
++(def xs (map (fn [x] (reset! calls (inc (deref calls))) x) (range 1000)))
++(println "built, calls so far:" (deref calls))
++(def three (doall (take 3 xs)))
++(println "took" three "- calls:" (deref calls))
++(println "took" (doall (take 3 xs)) "- calls:" (deref calls) "(memoized)")
++
++;; composing lazily builds no intermediate collections
++(println (reduce + 0 (take 10 (filter odd? (map (fn [x] (* x x)) (range))))))
++
++;; destructuring walks only as far as the pattern needs
++(let [[a b & more] (range)]
++  (println a b (take 3 more)))
++
++;; finite collections behave exactly as before
++(println (map inc [1 2 3]) (filter even? [1 2 3 4]))
++(println (range 5) (range 2 8 2) (range 5 0 -1))
++(println (count (range 100)) (empty? (range 0)) (seq (range 0)) (seq (range 2)))
++(println (= (range 3) [0 1 2]) (= (map inc [0 1]) (list 1 2)))
++(println (vec (take 3 (range))) (sort (take 4 (map (fn [x] (- 9 x)) (range)))))
++(println (cons 0 (range 3)) (rest (range 3)) (next (range 1)) (last (take 4 (range))))
++(doseq [x (take 3 (map inc (range)))] (print x ""))
++(println)
++
++;; realizing a long seq is iterative: no stack growth, no deep teardown
++(println (count (take 200000 (range))) (nth (iterate inc 0) 200000))

modified justfile +2 -1 Viewed

@@ -26,9 +26,10 @@ emit file: build
26	26	compile file: build
27	27	./{{bin}} build {{file}}
28	28
29		-# Persistent-collection benchmark: conj/assoc/get at scale
	29	+# Benchmarks: persistent-collection writes, and lazy early exit
30	30	bench: release
31	31	./{{bin}} run examples/persistent-bench.clj -d
	32	+ ./{{bin}} run examples/lazy-bench.clj -d
32	33
33	34	# Re-record tests/<name>.expected from current output
34	35	accept: build

  compile file: build
      ./{{bin}} build {{file}}
--# Persistent-collection benchmark: conj/assoc/get at scale
++# Benchmarks: persistent-collection writes, and lazy early exit
  bench: release
      ./{{bin}} run examples/persistent-bench.clj -d
++    ./{{bin}} run examples/lazy-bench.clj -d
  # Re-record tests/<name>.expected from current output
  accept: build

modified src/compiler.nim +5 -3 Viewed

@@ -101,6 +101,7 @@ proc quoteLit(v: Value): string =
101	101	var parts: seq[string] = @[]
102	102	for (k, val) in v.pairs: parts.add "(" & quoteLit(k) & ", " & quoteLit(val) & ")"
103	103	"mkMap(@[" & parts.join(", ") & "])"
	104	+ of kCons, kLazy: err("Can't quote a lazy seq")
104	105	of kFn: err("Can't quote a function")
105	106
106	107	proc emptySeqFix(s: string, elemType: string): string =
@@ -240,8 +241,7 @@ proc genLet(bindings: Value, body: seq[Value], dst: string, env: Env, c: Ctx) =
240	241	if isSym(p, "&"):
241	242	let restSym = symName(target.items[j + 1])
242	243	let id = c.gensym("l" & mangle(restSym))
243		- c.line("var " & id & ": Value = mkList(toSeq(" & v & ")[min(" & $idx &
244		- ", toSeq(" & v & ").len) .. ^1])")
	244	+ c.line("var " & id & ": Value = seqDrop(" & v & ", " & $idx & ")")
245	245	lenv.locals[restSym] = id
246	246	break
247	247	let id = c.gensym("l" & mangle(symName(p)))
@@ -331,6 +331,8 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
331	331	(if parts.len == 0: "newSeq[(Value, Value)]()" else: "@[" & parts.join(", ") & "]") & ")")
332	332	of kFn:
333	333	err("Can't emit a function literal")
	334	+ of kCons, kLazy:
	335	+ err("Can't emit a lazy seq literal")
334	336	of kList:
335	337	if f.items.len == 0:
336	338	c.line(dst & " = mkList(newSeq[Value]())"); return
@@ -514,7 +516,7 @@ proc genInto(f: Value, dst: string, env: Env, c: Ctx) =
514	516	let nm = symName(b.items[0])
515	517	let cv = genExpr(b.items[1], env, c)
516	518	let it = c.gensym("it")
517		- c.line("for " & it & " in toSeq(" & cv & "):")
	519	+ c.line("for " & it & " in elems(" & cv & "):")
518	520	c.push
519	521	let benv = newEnv(env)
520	522	let id = c.gensym("l" & mangle(nm))

      var parts: seq[string] = @[]
      for (k, val) in v.pairs: parts.add "(" & quoteLit(k) & ", " & quoteLit(val) & ")"
      "mkMap(@[" & parts.join(", ") & "])"
++  of kCons, kLazy: err("Can't quote a lazy seq")
    of kFn: err("Can't quote a function")
  proc emptySeqFix(s: string, elemType: string): string =
          if isSym(p, "&"):
            let restSym = symName(target.items[j + 1])
            let id = c.gensym("l" & mangle(restSym))
--          c.line("var " & id & ": Value = mkList(toSeq(" & v & ")[min(" & $idx &
++          c.line("var " & id & ": Value = seqDrop(" & v & ", " & $idx & ")")
--                 ", toSeq(" & v & ").len) .. ^1])")
            lenv.locals[restSym] = id
            break
          let id = c.gensym("l" & mangle(symName(p)))
        (if parts.len == 0: "newSeq[(Value, Value)]()" else: "@[" & parts.join(", ") & "]") & ")")
    of kFn:
      err("Can't emit a function literal")
++  of kCons, kLazy:
++    err("Can't emit a lazy seq literal")
    of kList:
      if f.items.len == 0:
        c.line(dst & " = mkList(newSeq[Value]())"); return
          let nm = symName(b.items[0])
          let cv = genExpr(b.items[1], env, c)
          let it = c.gensym("it")
--        c.line("for " & it & " in toSeq(" & cv & "):")
++        c.line("for " & it & " in elems(" & cv & "):")
          c.push
          let benv = newEnv(env)
          let id = c.gensym("l" & mangle(nm))

modified src/core.nim +183 -86 Viewed

@@ -87,6 +87,133 @@ proc conjOne(coll, x: Value): Value =
87	87	else: err("conj on map needs a pair")
88	88	else: err("conj not supported on " & prStr(coll))
89	89
	90	+# ------------------------------------------------------------- lazy seqs
	91	+## Producers share one shape: capture a Cursor, and return a thunk that
	92	+## advances a copy of it, yields one cons cell, and hands the advanced copy
	93	+## to the next thunk. Because each thunk only ever takes one step, an infinite
	94	+## source costs exactly as much as the consumer asks for.
	95	+
	96	+proc lazyOf(c: Cursor): Value =
	97	+ ## The remainder of a cursor, as a lazy seq.
	98	+ let cur = c
	99	+ mkLazy(proc (): Value =
	100	+ var cc = cur
	101	+ if not hasNext(cc): return NilV
	102	+ let x = next(cc)
	103	+ mkCons(x, lazyOf(cc)))
	104	+
	105	+proc lazyMap(f: Value, c: Cursor): Value =
	106	+ let cur = c
	107	+ mkLazy(proc (): Value =
	108	+ var cc = cur
	109	+ if not hasNext(cc): return NilV
	110	+ let x = next(cc)
	111	+ mkCons(call(f, @[x]), lazyMap(f, cc)))
	112	+
	113	+proc lazyMapN(f: Value, cs: seq[Cursor]): Value =
	114	+ let curs = cs
	115	+ mkLazy(proc (): Value =
	116	+ var ccs = curs
	117	+ var args: seq[Value] = @[]
	118	+ for i in 0 ..< ccs.len:
	119	+ if not hasNext(ccs[i]): return NilV # stop at the shortest
	120	+ args.add next(ccs[i])
	121	+ mkCons(call(f, args), lazyMapN(f, ccs)))
	122	+
	123	+proc lazyMapIndexed(f: Value, i: int64, c: Cursor): Value =
	124	+ let cur = c
	125	+ mkLazy(proc (): Value =
	126	+ var cc = cur
	127	+ if not hasNext(cc): return NilV
	128	+ let x = next(cc)
	129	+ mkCons(call(f, @[mkInt(i), x]), lazyMapIndexed(f, i + 1, cc)))
	130	+
	131	+proc lazyFilter(pred: Value, c: Cursor, keep: bool): Value =
	132	+ let cur = c
	133	+ mkLazy(proc (): Value =
	134	+ var cc = cur
	135	+ while hasNext(cc):
	136	+ let x = next(cc)
	137	+ if truthy(call(pred, @[x])) == keep:
	138	+ return mkCons(x, lazyFilter(pred, cc, keep))
	139	+ NilV)
	140	+
	141	+proc lazyTake(n: int, c: Cursor): Value =
	142	+ if n <= 0: return mkList(@[])
	143	+ let cur = c
	144	+ mkLazy(proc (): Value =
	145	+ var cc = cur
	146	+ if not hasNext(cc): return NilV
	147	+ let x = next(cc)
	148	+ mkCons(x, lazyTake(n - 1, cc)))
	149	+
	150	+proc lazyDrop(n: int, c: Cursor): Value =
	151	+ let cur = c
	152	+ mkLazy(proc (): Value =
	153	+ var cc = cur
	154	+ var k = n
	155	+ while k > 0 and hasNext(cc): discard next(cc); dec k
	156	+ force(lazyOf(cc)))
	157	+
	158	+proc lazyTakeWhile(pred: Value, c: Cursor): Value =
	159	+ let cur = c
	160	+ mkLazy(proc (): Value =
	161	+ var cc = cur
	162	+ if not hasNext(cc): return NilV
	163	+ let x = next(cc)
	164	+ if not truthy(call(pred, @[x])): return NilV
	165	+ mkCons(x, lazyTakeWhile(pred, cc)))
	166	+
	167	+proc lazyDropWhile(pred: Value, c: Cursor): Value =
	168	+ let cur = c
	169	+ mkLazy(proc (): Value =
	170	+ var cc = cur
	171	+ while true:
	172	+ var peek = cc
	173	+ if not hasNext(peek): return NilV
	174	+ let x = next(peek)
	175	+ if not truthy(call(pred, @[x])): return mkCons(x, lazyOf(peek))
	176	+ cc = peek)
	177	+
	178	+proc lazyRange(i, hi, step: int64, bounded: bool): Value =
	179	+ mkLazy(proc (): Value =
	180	+ if bounded and ((step > 0 and i >= hi) or (step < 0 and i <= hi)): return NilV
	181	+ mkCons(mkInt(i), lazyRange(i + step, hi, step, bounded)))
	182	+
	183	+proc lazyIterate(f, x: Value): Value =
	184	+ mkLazy(proc (): Value = mkCons(x, lazyIterate(f, call(f, @[x]))))
	185	+
	186	+proc lazyRepeat(x: Value, n: int64, bounded: bool): Value =
	187	+ mkLazy(proc (): Value =
	188	+ if bounded and n <= 0: return NilV
	189	+ mkCons(x, lazyRepeat(x, n - 1, bounded)))
	190	+
	191	+proc lazyRepeatedly(f: Value, n: int64, bounded: bool): Value =
	192	+ mkLazy(proc (): Value =
	193	+ if bounded and n <= 0: return NilV
	194	+ mkCons(call(f, @[]), lazyRepeatedly(f, n - 1, bounded)))
	195	+
	196	+proc lazyCycle(orig: Value, c: Cursor): Value =
	197	+ let cur = c
	198	+ mkLazy(proc (): Value =
	199	+ var cc = cur
	200	+ if not hasNext(cc):
	201	+ cc = cursor(orig) # wrap around
	202	+ if not hasNext(cc): return NilV # empty source: empty cycle
	203	+ let x = next(cc)
	204	+ mkCons(x, lazyCycle(orig, cc)))
	205	+
	206	+proc lazyConcat(colls: seq[Value], i: int, c: Cursor): Value =
	207	+ let cur = c
	208	+ mkLazy(proc (): Value =
	209	+ var cc = cur
	210	+ var k = i
	211	+ while not hasNext(cc):
	212	+ if k >= colls.len: return NilV
	213	+ cc = cursor(colls[k]); inc k
	214	+ let x = next(cc)
	215	+ mkCons(x, lazyConcat(colls, k, cc)))
	216	+
90	217	proc def(name: string, f: proc (args: seq[Value]): Value {.closure.}) =
91	218	setVar(name, mkFn(name, f))
92	219
@@ -165,7 +292,7 @@ proc registerCore*() =
165	292	def "coll?", proc (a: seq[Value]): Value =
166	293	mkBool(a[0].kind in {kList, kVector, kMap, kSet})
167	294	def "fn?", proc (a: seq[Value]): Value = mkBool(a[0].kind == kFn)
168		- def "empty?", proc (a: seq[Value]): Value = mkBool(count(a[0]) == 0)
	295	+ def "empty?", proc (a: seq[Value]): Value = mkBool(seqIsEmpty(a[0]))
169	296	def "contains?", proc (a: seq[Value]): Value =
170	297	let c = a[0]
171	298	if c.isNil or c.kind == kNil: return FalseV
@@ -221,7 +348,7 @@ proc registerCore*() =
221	348	let sep = (if a.len > 1: str(a[0]) else: "")
222	349	let coll = (if a.len > 1: a[1] else: a[0])
223	350	var parts: seq[string] = @[]
224		- for x in toSeq(coll): parts.add str(x)
	351	+ for x in elems(coll): parts.add str(x)
225	352	mkStr(parts.join(sep))
226	353	def "read-line", proc (a: seq[Value]): Value =
227	354	try: mkStr(stdin.readLine()) except CatchableError: NilV
@@ -243,34 +370,29 @@ proc registerCore*() =
243	370	def "set", proc (a: seq[Value]): Value = mkSet(toSeq(a[0]))
244	371	def "vec", proc (a: seq[Value]): Value = mkVector(toSeq(a[0]))
245	372	def "seq", proc (a: seq[Value]): Value =
246		- let s = toSeq(a[0])
247		- (if s.len == 0: NilV else: mkList(s))
	373	+ # does not realize a lazy seq — just asks whether it has a first element
	374	+ (if seqIsEmpty(a[0]): NilV else: a[0])
248	375	def "count", proc (a: seq[Value]): Value =
249	376	if a[0].isNil or a[0].kind == kNil: return mkInt(0)
250	377	mkInt(count(a[0]))
251	378	def "conj", proc (a: seq[Value]): Value =
252	379	result = a[0]
253	380	for i in 1 ..< a.len: result = conjOne(result, a[i])
254		- def "cons", proc (a: seq[Value]): Value = mkList(@[a[0]] & toSeq(a[1]))
	381	+ def "cons", proc (a: seq[Value]): Value = mkCons(a[0], a[1])
255	382	def "first", proc (a: seq[Value]): Value =
256	383	if a[0].kind == kVector:
257	384	return (if a[0].vec.cnt == 0: NilV else: vecNth(a[0].vec, 0))
258		- let s = toSeq(a[0])
259		- (if s.len == 0: NilV else: s[0])
260		- def "second", proc (a: seq[Value]): Value =
261		- let s = toSeq(a[0])
262		- (if s.len < 2: NilV else: s[1])
	385	+ seqFirst(a[0])
	386	+ def "second", proc (a: seq[Value]): Value = seqFirst(seqRest(a[0]))
263	387	def "last", proc (a: seq[Value]): Value =
264	388	if a[0].kind == kVector:
265	389	return (if a[0].vec.cnt == 0: NilV else: vecNth(a[0].vec, a[0].vec.cnt - 1))
266		- let s = toSeq(a[0])
267		- (if s.len == 0: NilV else: s[^1])
268		- def "rest", proc (a: seq[Value]): Value =
269		- let s = toSeq(a[0])
270		- (if s.len <= 1: mkList(@[]) else: mkList(s[1 .. ^1]))
	390	+ result = NilV
	391	+ for x in elems(a[0]): result = x
	392	+ def "rest", proc (a: seq[Value]): Value = seqRest(a[0])
271	393	def "next", proc (a: seq[Value]): Value =
272		- let s = toSeq(a[0])
273		- (if s.len <= 1: NilV else: mkList(s[1 .. ^1]))
	394	+ let r = seqRest(a[0])
	395	+ (if seqIsEmpty(r): NilV else: r)
274	396	def "nth", proc (a: seq[Value]): Value =
275	397	let i = int(intOf(a[1]))
276	398	if a[0].kind == kVector:
@@ -278,9 +400,15 @@ proc registerCore*() =
278	400	if i >= 0 and i < a[0].vec.cnt: return vecNth(a[0].vec, i)
279	401	if a.len > 2: return a[2]
280	402	err("Index out of bounds: " & $i)
281		- let s = toSeq(a[0])
282		- if i >= 0 and i < s.len: s[i]
283		- elif a.len > 2: a[2]
	403	+ if i >= 0:
	404	+ # walks the seq, realizing no more of it than the index demands
	405	+ var k = i
	406	+ var c = cursor(a[0])
	407	+ while hasNext(c):
	408	+ let x = next(c)
	409	+ if k == 0: return x
	410	+ dec k
	411	+ if a.len > 2: a[2]
284	412	else: err("Index out of bounds: " & $i)
285	413	def "get", proc (a: seq[Value]): Value =
286	414	getIn(a[0], a[1], (if a.len > 2: a[2] else: NilV))
@@ -310,7 +438,7 @@ proc registerCore*() =
310	438	for e in mapEntries(a[0].m): r.add e.val
311	439	(if r.len == 0: NilV else: mkList(r))
312	440	def "reverse", proc (a: seq[Value]): Value =
313		- var s = toSeq(a[0])
	441	+ let s = toSeq(a[0])
314	442	var r: seq[Value] = @[]
315	443	for i in countdown(s.len - 1, 0): r.add s[i]
316	444	mkList(r)
@@ -322,26 +450,26 @@ proc registerCore*() =
322	450	elif a.len >= 2:
323	451	lo = intOf(a[0]); hi = intOf(a[1])
324	452	if a.len > 2: step = intOf(a[2])
325		- var r: seq[Value] = @[]
326		- if step > 0:
327		- var i = lo
328		- while i < hi: r.add mkInt(i); i += step
329		- elif step < 0:
330		- var i = lo
331		- while i > hi: r.add mkInt(i); i += step
332		- mkList(r)
	453	+ # (range) with no bound is infinite; everything else stops at hi
	454	+ lazyRange(lo, hi, step, bounded = a.len > 0)
333	455	def "take", proc (a: seq[Value]): Value =
334		- let n = int(intOf(a[0]))
335		- let s = toSeq(a[1])
336		- mkList(s[0 ..< min(n, s.len)])
	456	+ lazyTake(int(intOf(a[0])), cursor(a[1]))
337	457	def "drop", proc (a: seq[Value]): Value =
338		- let n = int(intOf(a[0]))
339		- let s = toSeq(a[1])
340		- (if n >= s.len: mkList(@[]) else: mkList(s[n .. ^1]))
	458	+ lazyDrop(int(intOf(a[0])), cursor(a[1]))
341	459	def "concat", proc (a: seq[Value]): Value =
342		- var r: seq[Value] = @[]
343		- for x in a: r.add toSeq(x)
344		- mkList(r)
	460	+ lazyConcat(a, 0, cursor(NilV))
	461	+ def "iterate", proc (a: seq[Value]): Value = lazyIterate(a[0], a[1])
	462	+ def "repeat", proc (a: seq[Value]): Value =
	463	+ (if a.len == 1: lazyRepeat(a[0], 0, bounded = false)
	464	+ else: lazyRepeat(a[1], intOf(a[0]), bounded = true))
	465	+ def "repeatedly", proc (a: seq[Value]): Value =
	466	+ (if a.len == 1: lazyRepeatedly(a[0], 0, bounded = false)
	467	+ else: lazyRepeatedly(a[1], intOf(a[0]), bounded = true))
	468	+ def "cycle", proc (a: seq[Value]): Value = lazyCycle(a[0], cursor(a[0]))
	469	+ def "doall", proc (a: seq[Value]): Value = mkList(toSeq(a[0]))
	470	+ def "dorun", proc (a: seq[Value]): Value =
	471	+ for x in elems(a[0]): discard
	472	+ NilV
345	473	def "sort", proc (a: seq[Value]): Value =
346	474	var s = toSeq(a[^1])
347	475	let cmpFn = (if a.len > 1: a[0] else: NilV)
@@ -370,7 +498,7 @@ proc registerCore*() =
370	498	mkList(s)
371	499	def "distinct", proc (a: seq[Value]): Value =
372	500	var r: seq[Value] = @[]
373		- for x in toSeq(a[0]):
	501	+ for x in elems(a[0]):
374	502	var dup = false
375	503	for y in r:
376	504	if equals(x, y): dup = true; break
@@ -378,7 +506,7 @@ proc registerCore*() =
378	506	mkList(r)
379	507	def "interpose", proc (a: seq[Value]): Value =
380	508	var r: seq[Value] = @[]
381		- for x in toSeq(a[1]):
	509	+ for x in elems(a[1]):
382	510	if r.len > 0: r.add a[0]
383	511	r.add x
384	512	mkList(r)
@@ -398,41 +526,20 @@ proc registerCore*() =
398	526	callArgs.add toSeq(a[^1])
399	527	call(a[0], callArgs)
400	528	def "map", proc (a: seq[Value]): Value =
401		- let f = a[0]
402		- if a.len == 2:
403		- var r: seq[Value] = @[]
404		- for x in toSeq(a[1]): r.add call(f, @[x])
405		- return mkList(r)
406		- var colls: seq[seq[Value]] = @[]
407		- for i in 1 ..< a.len: colls.add toSeq(a[i])
408		- var n = colls[0].len
409		- for c in colls: n = min(n, c.len)
410		- var r: seq[Value] = @[]
411		- for i in 0 ..< n:
412		- var args: seq[Value] = @[]
413		- for c in colls: args.add c[i]
414		- r.add call(f, args)
415		- mkList(r)
	529	+ if a.len == 2: return lazyMap(a[0], cursor(a[1]))
	530	+ var cs: seq[Cursor] = @[]
	531	+ for i in 1 ..< a.len: cs.add cursor(a[i])
	532	+ lazyMapN(a[0], cs)
416	533	def "mapv", proc (a: seq[Value]): Value =
417	534	var r: seq[Value] = @[]
418		- for x in toSeq(a[1]): r.add call(a[0], @[x])
	535	+ for x in elems(a[1]): r.add call(a[0], @[x])
419	536	mkVector(r)
420	537	def "map-indexed", proc (a: seq[Value]): Value =
421		- var r: seq[Value] = @[]
422		- var i = 0
423		- for x in toSeq(a[1]):
424		- r.add call(a[0], @[mkInt(i), x]); inc i
425		- mkList(r)
	538	+ lazyMapIndexed(a[0], 0, cursor(a[1]))
426	539	def "filter", proc (a: seq[Value]): Value =
427		- var r: seq[Value] = @[]
428		- for x in toSeq(a[1]):
429		- if truthy(call(a[0], @[x])): r.add x
430		- mkList(r)
	540	+ lazyFilter(a[0], cursor(a[1]), keep = true)
431	541	def "remove", proc (a: seq[Value]): Value =
432		- var r: seq[Value] = @[]
433		- for x in toSeq(a[1]):
434		- if not truthy(call(a[0], @[x])): r.add x
435		- mkList(r)
	542	+ lazyFilter(a[0], cursor(a[1]), keep = false)
436	543	def "reduce", proc (a: seq[Value]): Value =
437	544	let f = a[0]
438	545	if a.len == 2:
@@ -442,40 +549,30 @@ proc registerCore*() =
442	549	for i in 1 ..< s.len: acc = call(f, @[acc, s[i]])
443	550	return acc
444	551	var acc = a[1]
445		- for x in toSeq(a[2]): acc = call(f, @[acc, x])
	552	+ for x in elems(a[2]): acc = call(f, @[acc, x])
446	553	acc
447	554	def "some", proc (a: seq[Value]): Value =
448		- for x in toSeq(a[1]):
	555	+ for x in elems(a[1]):
449	556	let r = call(a[0], @[x])
450	557	if truthy(r): return r
451	558	NilV
452	559	def "every?", proc (a: seq[Value]): Value =
453		- for x in toSeq(a[1]):
	560	+ for x in elems(a[1]):
454	561	if not truthy(call(a[0], @[x])): return FalseV
455	562	TrueV
456	563	def "take-while", proc (a: seq[Value]): Value =
457		- var r: seq[Value] = @[]
458		- for x in toSeq(a[1]):
459		- if not truthy(call(a[0], @[x])): break
460		- r.add x
461		- mkList(r)
	564	+ lazyTakeWhile(a[0], cursor(a[1]))
462	565	def "drop-while", proc (a: seq[Value]): Value =
463		- var r: seq[Value] = @[]
464		- var dropping = true
465		- for x in toSeq(a[1]):
466		- if dropping and truthy(call(a[0], @[x])): continue
467		- dropping = false
468		- r.add x
469		- mkList(r)
	566	+ lazyDropWhile(a[0], cursor(a[1]))
470	567	def "group-by", proc (a: seq[Value]): Value =
471	568	var m = emptyPMap()
472		- for x in toSeq(a[1]):
	569	+ for x in elems(a[1]):
473	570	let k = call(a[0], @[x])
474	571	m = mapAssoc(m, k, conjOne(mapGet(m, k, mkVector(@[])), x))
475	572	mkMapOf(m)
476	573	def "frequencies", proc (a: seq[Value]): Value =
477	574	var m = emptyPMap()
478		- for x in toSeq(a[0]):
	575	+ for x in elems(a[0]):
479	576	m = mapAssoc(m, x, mkInt(mapGet(m, x, mkInt(0)).i + 1))
480	577	mkMapOf(m)
481	578	def "identity", proc (a: seq[Value]): Value = a[0]

      else: err("conj on map needs a pair")
    else: err("conj not supported on " & prStr(coll))
++# ------------------------------------------------------------- lazy seqs
++## Producers share one shape: capture a Cursor, and return a thunk that
++## advances a *copy* of it, yields one cons cell, and hands the advanced copy
++## to the next thunk. Because each thunk only ever takes one step, an infinite
++## source costs exactly as much as the consumer asks for.
++
++proc lazyOf(c: Cursor): Value =
++  ## The remainder of a cursor, as a lazy seq.
++  let cur = c
++  mkLazy(proc (): Value =
++    var cc = cur
++    if not hasNext(cc): return NilV
++    let x = next(cc)
++    mkCons(x, lazyOf(cc)))
++
++proc lazyMap(f: Value, c: Cursor): Value =
++  let cur = c
++  mkLazy(proc (): Value =
++    var cc = cur
++    if not hasNext(cc): return NilV
++    let x = next(cc)
++    mkCons(call(f, @[x]), lazyMap(f, cc)))
++
++proc lazyMapN(f: Value, cs: seq[Cursor]): Value =
++  let curs = cs
++  mkLazy(proc (): Value =
++    var ccs = curs
++    var args: seq[Value] = @[]
++    for i in 0 ..< ccs.len:
++      if not hasNext(ccs[i]): return NilV    # stop at the shortest
++      args.add next(ccs[i])
++    mkCons(call(f, args), lazyMapN(f, ccs)))
++
++proc lazyMapIndexed(f: Value, i: int64, c: Cursor): Value =
++  let cur = c
++  mkLazy(proc (): Value =
++    var cc = cur
++    if not hasNext(cc): return NilV
++    let x = next(cc)
++    mkCons(call(f, @[mkInt(i), x]), lazyMapIndexed(f, i + 1, cc)))
++
++proc lazyFilter(pred: Value, c: Cursor, keep: bool): Value =
++  let cur = c
++  mkLazy(proc (): Value =
++    var cc = cur
++    while hasNext(cc):
++      let x = next(cc)
++      if truthy(call(pred, @[x])) == keep:
++        return mkCons(x, lazyFilter(pred, cc, keep))
++    NilV)
++
++proc lazyTake(n: int, c: Cursor): Value =
++  if n <= 0: return mkList(@[])
++  let cur = c
++  mkLazy(proc (): Value =
++    var cc = cur
++    if not hasNext(cc): return NilV
++    let x = next(cc)
++    mkCons(x, lazyTake(n - 1, cc)))
++
++proc lazyDrop(n: int, c: Cursor): Value =
++  let cur = c
++  mkLazy(proc (): Value =
++    var cc = cur
++    var k = n
++    while k > 0 and hasNext(cc): discard next(cc); dec k
++    force(lazyOf(cc)))
++
++proc lazyTakeWhile(pred: Value, c: Cursor): Value =
++  let cur = c
++  mkLazy(proc (): Value =
++    var cc = cur
++    if not hasNext(cc): return NilV
++    let x = next(cc)
++    if not truthy(call(pred, @[x])): return NilV
++    mkCons(x, lazyTakeWhile(pred, cc)))
++
++proc lazyDropWhile(pred: Value, c: Cursor): Value =
++  let cur = c
++  mkLazy(proc (): Value =
++    var cc = cur
++    while true:
++      var peek = cc
++      if not hasNext(peek): return NilV
++      let x = next(peek)
++      if not truthy(call(pred, @[x])): return mkCons(x, lazyOf(peek))
++      cc = peek)
++
++proc lazyRange(i, hi, step: int64, bounded: bool): Value =
++  mkLazy(proc (): Value =
++    if bounded and ((step > 0 and i >= hi) or (step < 0 and i <= hi)): return NilV
++    mkCons(mkInt(i), lazyRange(i + step, hi, step, bounded)))
++
++proc lazyIterate(f, x: Value): Value =
++  mkLazy(proc (): Value = mkCons(x, lazyIterate(f, call(f, @[x]))))
++
++proc lazyRepeat(x: Value, n: int64, bounded: bool): Value =
++  mkLazy(proc (): Value =
++    if bounded and n <= 0: return NilV
++    mkCons(x, lazyRepeat(x, n - 1, bounded)))
++
++proc lazyRepeatedly(f: Value, n: int64, bounded: bool): Value =
++  mkLazy(proc (): Value =
++    if bounded and n <= 0: return NilV
++    mkCons(call(f, @[]), lazyRepeatedly(f, n - 1, bounded)))
++
++proc lazyCycle(orig: Value, c: Cursor): Value =
++  let cur = c
++  mkLazy(proc (): Value =
++    var cc = cur
++    if not hasNext(cc):
++      cc = cursor(orig)                      # wrap around
++      if not hasNext(cc): return NilV        # empty source: empty cycle
++    let x = next(cc)
++    mkCons(x, lazyCycle(orig, cc)))
++
++proc lazyConcat(colls: seq[Value], i: int, c: Cursor): Value =
++  let cur = c
++  mkLazy(proc (): Value =
++    var cc = cur
++    var k = i
++    while not hasNext(cc):
++      if k >= colls.len: return NilV
++      cc = cursor(colls[k]); inc k
++    let x = next(cc)
++    mkCons(x, lazyConcat(colls, k, cc)))
++
  proc def(name: string, f: proc (args: seq[Value]): Value {.closure.}) =
    setVar(name, mkFn(name, f))
    def "coll?", proc (a: seq[Value]): Value =
      mkBool(a[0].kind in {kList, kVector, kMap, kSet})
    def "fn?", proc (a: seq[Value]): Value = mkBool(a[0].kind == kFn)
--  def "empty?", proc (a: seq[Value]): Value = mkBool(count(a[0]) == 0)
++  def "empty?", proc (a: seq[Value]): Value = mkBool(seqIsEmpty(a[0]))
    def "contains?", proc (a: seq[Value]): Value =
      let c = a[0]
      if c.isNil or c.kind == kNil: return FalseV
      let sep = (if a.len > 1: str(a[0]) else: "")
      let coll = (if a.len > 1: a[1] else: a[0])
      var parts: seq[string] = @[]
--    for x in toSeq(coll): parts.add str(x)
++    for x in elems(coll): parts.add str(x)
      mkStr(parts.join(sep))
    def "read-line", proc (a: seq[Value]): Value =
      try: mkStr(stdin.readLine()) except CatchableError: NilV
    def "set", proc (a: seq[Value]): Value = mkSet(toSeq(a[0]))
    def "vec", proc (a: seq[Value]): Value = mkVector(toSeq(a[0]))
    def "seq", proc (a: seq[Value]): Value =
--    let s = toSeq(a[0])
++    # does not realize a lazy seq — just asks whether it has a first element
--    (if s.len == 0: NilV else: mkList(s))
++    (if seqIsEmpty(a[0]): NilV else: a[0])
    def "count", proc (a: seq[Value]): Value =
      if a[0].isNil or a[0].kind == kNil: return mkInt(0)
      mkInt(count(a[0]))
    def "conj", proc (a: seq[Value]): Value =
      result = a[0]
      for i in 1 ..< a.len: result = conjOne(result, a[i])
--  def "cons", proc (a: seq[Value]): Value = mkList(@[a[0]] & toSeq(a[1]))
++  def "cons", proc (a: seq[Value]): Value = mkCons(a[0], a[1])
    def "first", proc (a: seq[Value]): Value =
      if a[0].kind == kVector:
        return (if a[0].vec.cnt == 0: NilV else: vecNth(a[0].vec, 0))
--    let s = toSeq(a[0])
++    seqFirst(a[0])
--    (if s.len == 0: NilV else: s[0])
++  def "second", proc (a: seq[Value]): Value = seqFirst(seqRest(a[0]))
--  def "second", proc (a: seq[Value]): Value =
--    let s = toSeq(a[0])
--    (if s.len < 2: NilV else: s[1])
    def "last", proc (a: seq[Value]): Value =
      if a[0].kind == kVector:
        return (if a[0].vec.cnt == 0: NilV else: vecNth(a[0].vec, a[0].vec.cnt - 1))
--    let s = toSeq(a[0])
++    result = NilV
--    (if s.len == 0: NilV else: s[^1])
++    for x in elems(a[0]): result = x
--  def "rest", proc (a: seq[Value]): Value =
++  def "rest", proc (a: seq[Value]): Value = seqRest(a[0])
--    let s = toSeq(a[0])
--    (if s.len <= 1: mkList(@[]) else: mkList(s[1 .. ^1]))
    def "next", proc (a: seq[Value]): Value =
--    let s = toSeq(a[0])
++    let r = seqRest(a[0])
--    (if s.len <= 1: NilV else: mkList(s[1 .. ^1]))
++    (if seqIsEmpty(r): NilV else: r)
    def "nth", proc (a: seq[Value]): Value =
      let i = int(intOf(a[1]))
      if a[0].kind == kVector:
        if i >= 0 and i < a[0].vec.cnt: return vecNth(a[0].vec, i)
        if a.len > 2: return a[2]
        err("Index out of bounds: " & $i)
--    let s = toSeq(a[0])
++    if i >= 0:
--    if i >= 0 and i < s.len: s[i]
++      # walks the seq, realizing no more of it than the index demands
--    elif a.len > 2: a[2]
++      var k = i
++      var c = cursor(a[0])
++      while hasNext(c):
++        let x = next(c)
++        if k == 0: return x
++        dec k
++    if a.len > 2: a[2]
      else: err("Index out of bounds: " & $i)
    def "get", proc (a: seq[Value]): Value =
      getIn(a[0], a[1], (if a.len > 2: a[2] else: NilV))
      for e in mapEntries(a[0].m): r.add e.val
      (if r.len == 0: NilV else: mkList(r))
    def "reverse", proc (a: seq[Value]): Value =
--    var s = toSeq(a[0])
++    let s = toSeq(a[0])
      var r: seq[Value] = @[]
      for i in countdown(s.len - 1, 0): r.add s[i]
      mkList(r)
      elif a.len >= 2:
        lo = intOf(a[0]); hi = intOf(a[1])
        if a.len > 2: step = intOf(a[2])
--    var r: seq[Value] = @[]
++    # (range) with no bound is infinite; everything else stops at hi
--    if step > 0:
++    lazyRange(lo, hi, step, bounded = a.len > 0)
--      var i = lo
--      while i < hi: r.add mkInt(i); i += step
--    elif step < 0:
--      var i = lo
--      while i > hi: r.add mkInt(i); i += step
--    mkList(r)
    def "take", proc (a: seq[Value]): Value =
--    let n = int(intOf(a[0]))
++    lazyTake(int(intOf(a[0])), cursor(a[1]))
--    let s = toSeq(a[1])
--    mkList(s[0 ..< min(n, s.len)])
    def "drop", proc (a: seq[Value]): Value =
--    let n = int(intOf(a[0]))
++    lazyDrop(int(intOf(a[0])), cursor(a[1]))
--    let s = toSeq(a[1])
--    (if n >= s.len: mkList(@[]) else: mkList(s[n .. ^1]))
    def "concat", proc (a: seq[Value]): Value =
--    var r: seq[Value] = @[]
++    lazyConcat(a, 0, cursor(NilV))
--    for x in a: r.add toSeq(x)
++  def "iterate", proc (a: seq[Value]): Value = lazyIterate(a[0], a[1])
--    mkList(r)
++  def "repeat", proc (a: seq[Value]): Value =
++    (if a.len == 1: lazyRepeat(a[0], 0, bounded = false)
++     else: lazyRepeat(a[1], intOf(a[0]), bounded = true))
++  def "repeatedly", proc (a: seq[Value]): Value =
++    (if a.len == 1: lazyRepeatedly(a[0], 0, bounded = false)
++     else: lazyRepeatedly(a[1], intOf(a[0]), bounded = true))
++  def "cycle", proc (a: seq[Value]): Value = lazyCycle(a[0], cursor(a[0]))
++  def "doall", proc (a: seq[Value]): Value = mkList(toSeq(a[0]))
++  def "dorun", proc (a: seq[Value]): Value =
++    for x in elems(a[0]): discard
++    NilV
    def "sort", proc (a: seq[Value]): Value =
      var s = toSeq(a[^1])
      let cmpFn = (if a.len > 1: a[0] else: NilV)
      mkList(s)
    def "distinct", proc (a: seq[Value]): Value =
      var r: seq[Value] = @[]
--    for x in toSeq(a[0]):
++    for x in elems(a[0]):
        var dup = false
        for y in r:
          if equals(x, y): dup = true; break
      mkList(r)
    def "interpose", proc (a: seq[Value]): Value =
      var r: seq[Value] = @[]
--    for x in toSeq(a[1]):
++    for x in elems(a[1]):
        if r.len > 0: r.add a[0]
        r.add x
      mkList(r)
      callArgs.add toSeq(a[^1])
      call(a[0], callArgs)
    def "map", proc (a: seq[Value]): Value =
--    let f = a[0]
++    if a.len == 2: return lazyMap(a[0], cursor(a[1]))
--    if a.len == 2:
++    var cs: seq[Cursor] = @[]
--      var r: seq[Value] = @[]
++    for i in 1 ..< a.len: cs.add cursor(a[i])
--      for x in toSeq(a[1]): r.add call(f, @[x])
++    lazyMapN(a[0], cs)
--      return mkList(r)
--    var colls: seq[seq[Value]] = @[]
--    for i in 1 ..< a.len: colls.add toSeq(a[i])
--    var n = colls[0].len
--    for c in colls: n = min(n, c.len)
--    var r: seq[Value] = @[]
--    for i in 0 ..< n:
--      var args: seq[Value] = @[]
--      for c in colls: args.add c[i]
--      r.add call(f, args)
--    mkList(r)
    def "mapv", proc (a: seq[Value]): Value =
      var r: seq[Value] = @[]
--    for x in toSeq(a[1]): r.add call(a[0], @[x])
++    for x in elems(a[1]): r.add call(a[0], @[x])
      mkVector(r)
    def "map-indexed", proc (a: seq[Value]): Value =
--    var r: seq[Value] = @[]
++    lazyMapIndexed(a[0], 0, cursor(a[1]))
--    var i = 0
--    for x in toSeq(a[1]):
--      r.add call(a[0], @[mkInt(i), x]); inc i
--    mkList(r)
    def "filter", proc (a: seq[Value]): Value =
--    var r: seq[Value] = @[]
++    lazyFilter(a[0], cursor(a[1]), keep = true)
--    for x in toSeq(a[1]):
--      if truthy(call(a[0], @[x])): r.add x
--    mkList(r)
    def "remove", proc (a: seq[Value]): Value =
--    var r: seq[Value] = @[]
++    lazyFilter(a[0], cursor(a[1]), keep = false)
--    for x in toSeq(a[1]):
--      if not truthy(call(a[0], @[x])): r.add x
--    mkList(r)
    def "reduce", proc (a: seq[Value]): Value =
      let f = a[0]
      if a.len == 2:
        for i in 1 ..< s.len: acc = call(f, @[acc, s[i]])
        return acc
      var acc = a[1]
--    for x in toSeq(a[2]): acc = call(f, @[acc, x])
++    for x in elems(a[2]): acc = call(f, @[acc, x])
      acc
    def "some", proc (a: seq[Value]): Value =
--    for x in toSeq(a[1]):
++    for x in elems(a[1]):
        let r = call(a[0], @[x])
        if truthy(r): return r
      NilV
    def "every?", proc (a: seq[Value]): Value =
--    for x in toSeq(a[1]):
++    for x in elems(a[1]):
        if not truthy(call(a[0], @[x])): return FalseV
      TrueV
    def "take-while", proc (a: seq[Value]): Value =
--    var r: seq[Value] = @[]
++    lazyTakeWhile(a[0], cursor(a[1]))
--    for x in toSeq(a[1]):
--      if not truthy(call(a[0], @[x])): break
--      r.add x
--    mkList(r)
    def "drop-while", proc (a: seq[Value]): Value =
--    var r: seq[Value] = @[]
++    lazyDropWhile(a[0], cursor(a[1]))
--    var dropping = true
--    for x in toSeq(a[1]):
--      if dropping and truthy(call(a[0], @[x])): continue
--      dropping = false
--      r.add x
--    mkList(r)
    def "group-by", proc (a: seq[Value]): Value =
      var m = emptyPMap()
--    for x in toSeq(a[1]):
++    for x in elems(a[1]):
        let k = call(a[0], @[x])
        m = mapAssoc(m, k, conjOne(mapGet(m, k, mkVector(@[])), x))
      mkMapOf(m)
    def "frequencies", proc (a: seq[Value]): Value =
      var m = emptyPMap()
--    for x in toSeq(a[0]):
++    for x in elems(a[0]):
        m = mapAssoc(m, x, mkInt(mapGet(m, x, mkInt(0)).i + 1))
      mkMapOf(m)
    def "identity", proc (a: seq[Value]): Value = a[0]

modified src/runtime.nim +201 -44 Viewed

@@ -17,7 +17,7 @@ const
17	17	type
18	18	Kind* = enum
19	19	kNil, kBool, kInt, kFloat, kStr, kKeyword, kSymbol,
20		- kList, kVector, kMap, kSet, kFn
	20	+ kList, kVector, kMap, kSet, kCons, kLazy, kFn
21	21
22	22	VNode* = ref object
23	23	## A trie node: leaves hold values, internal nodes hold children.
@@ -56,7 +56,8 @@ type
56	56	cnt*: int
57	57	nextOrd*: int
58	58
59		- Value* = ref object
	59	+ Value* = ref ValueObj
	60	+ ValueObj* = object
60	61	case kind*: Kind
61	62	of kNil: discard
62	63	of kBool: b*: bool
@@ -66,12 +67,58 @@ type
66	67	of kList: xs*: seq[Value]
67	68	of kVector: vec*: PVec
68	69	of kMap, kSet: m*: PMap
	70	+ of kCons:
	71	+ head*: Value
	72	+ tl*: Value ## rest of the seq: a cons, a lazy seq, a coll, or nil
	73	+ of kLazy:
	74	+ thunk*: proc (): Value {.closure.}
	75	+ cached*: Value
	76	+ forced*: bool
69	77	of kFn:
70	78	fn*: proc (args: seq[Value]): Value {.closure.}
71	79	name*: string
72	80
73	81	CljError* = object of CatchableError
74	82
	83	+# ------------------------------------------------------------ teardown
	84	+## A realized lazy seq is a chain of `cons -> lazy -> cons -> …` refs, and ARC
	85	+## frees a chain by recursing into it — a million-element seq means a million
	86	+## destructor frames, i.e. a segfault at scope exit. So `kCons`/`kLazy` hand
	87	+## their tail to a worklist instead of letting the field drop inline, and the
	88	+## outermost destructor drains it in a loop. Nothing shared is mutated: a node
	89	+## another seq still holds simply survives with its refcount intact.
	90	+##
	91	+## Defining `=destroy` means the compiler stops generating field teardown for
	92	+## `ValueObj`, so every branch below has to release its own fields.
	93	+
	94	+var pendingFree: seq[Value] = @[]
	95	+var draining = false
	96	+
	97	+proc `=destroy`*(x: var ValueObj) =
	98	+ case x.kind
	99	+ of kStr, kKeyword, kSymbol: `=destroy`(x.s)
	100	+ of kList: `=destroy`(x.xs)
	101	+ of kVector: `=destroy`(x.vec)
	102	+ of kMap, kSet: `=destroy`(x.m)
	103	+ of kFn:
	104	+ `=destroy`(x.fn)
	105	+ `=destroy`(x.name)
	106	+ of kCons:
	107	+ `=destroy`(x.head)
	108	+ if not x.tl.isNil: pendingFree.add x.tl # +1, outlives the release below
	109	+ `=destroy`(x.tl)
	110	+ of kLazy:
	111	+ `=destroy`(x.thunk)
	112	+ if not x.cached.isNil: pendingFree.add x.cached
	113	+ `=destroy`(x.cached)
	114	+ of kNil, kBool, kInt, kFloat: discard
	115	+ if draining: return
	116	+ draining = true
	117	+ while pendingFree.len > 0:
	118	+ let v = pendingFree.pop()
	119	+ discard v # dies here: its own tail is queued, not recursed into
	120	+ draining = false
	121	+
75	122	let NilV* = Value(kind: kNil)
76	123	let TrueV* = Value(kind: kBool, b: true)
77	124	let FalseV* = Value(kind: kBool, b: false)
@@ -81,6 +128,7 @@ proc err*(msg: string) {.noreturn.} = raise newException(CljError, msg)
81	128	proc equals*(a, b: Value): bool
82	129	proc hashValue*(v: Value): uint32
83	130	proc prStr*(v: Value): string
	131	+proc toSeq*(v: Value): seq[Value]
84	132
85	133	# ------------------------------------------------------- persistent vector
86	134	let emptyVNode = VNode(leaf: false, kids: @[])
@@ -322,36 +370,6 @@ proc mapEntries*(m: PMap): seq[MEntry] =
322	370	collect(m.root, result)
323	371	result.sort(proc (a, b: MEntry): int = cmp(a.ord, b.ord))
324	372
325		-proc hashValue*(v: Value): uint32 =
326		- if v.isNil: return 0
327		- case v.kind
328		- of kNil: 0'u32
329		- of kBool: (if v.b: 0x9e3779b9'u32 else: 0x85ebca6b'u32)
330		- of kInt: uint32(hash(v.i))
331		- of kFloat:
332		- # ints and floats compare equal across kinds, so they must hash alike
333		- if v.f == float64(int64(v.f)): uint32(hash(int64(v.f)))
334		- else: uint32(hash(v.f))
335		- of kStr: mixHash(1'u32, uint32(hash(v.s)))
336		- of kKeyword: mixHash(2'u32, uint32(hash(v.s)))
337		- of kSymbol: mixHash(3'u32, uint32(hash(v.s)))
338		- of kList, kVector:
339		- # lists and vectors are `=` when their elements are, so they hash alike
340		- var h = 7'u32
341		- for x in (if v.kind == kList: v.xs else: vecToSeq(v.vec)):
342		- h = mixHash(h, hashValue(x))
343		- h
344		- of kSet:
345		- var h = 0'u32 # xor: independent of iteration order
346		- for e in mapEntries(v.m): h = h xor hashValue(e.key)
347		- h
348		- of kMap:
349		- var h = 0'u32
350		- for e in mapEntries(v.m):
351		- h = h xor mixHash(hashValue(e.key), hashValue(e.val))
352		- h
353		- of kFn: uint32(hash(cast[int](cast[pointer](v))))
354		-
355	373	# ------------------------------------------------------------ constructors
356	374	proc mkBool*(x: bool): Value = (if x: TrueV else: FalseV)
357	375	proc mkInt*(x: int64): Value = Value(kind: kInt, i: x)
@@ -379,6 +397,131 @@ proc mkSet*(xs: seq[Value]): Value =
379	397	proc mkFn*(name: string, f: proc (args: seq[Value]): Value {.closure.}): Value =
380	398	Value(kind: kFn, fn: f, name: name)
381	399
	400	+# --------------------------------------------------------------- lazy seqs
	401	+## A lazy seq is a thunk that, when forced, yields either nil/`kNil` (the end)
	402	+## or a cons cell whose tail is usually another lazy seq. Forcing is memoized
	403	+## in place, so each element is computed once no matter how often it is walked.
	404	+## Nothing here recurses per element: `force` loops, and so does every producer
	405	+## in core, which is what keeps `(nth (iterate inc 0) 1000000)` from blowing the
	406	+## stack.
	407	+
	408	+proc mkCons*(h, t: Value): Value = Value(kind: kCons, head: h, tl: t)
	409	+
	410	+proc mkLazy*(f: proc (): Value {.closure.}): Value =
	411	+ Value(kind: kLazy, thunk: f, cached: nil, forced: false)
	412	+
	413	+proc force*(v: Value): Value =
	414	+ ## Realize one step: follow a chain of lazy seqs down to a cons, a concrete
	415	+ ## collection, or the end of the seq.
	416	+ var cur = v
	417	+ while not cur.isNil and cur.kind == kLazy:
	418	+ if not cur.forced:
	419	+ cur.cached = cur.thunk()
	420	+ cur.forced = true
	421	+ cur.thunk = nil # drop the closure so its captures can be collected
	422	+ cur = cur.cached
	423	+ cur
	424	+
	425	+proc isSeqNode(v: Value): bool =
	426	+ not v.isNil and v.kind in {kCons, kLazy}
	427	+
	428	+type Cursor* = object
	429	+ ## Walks any seqable value without materializing it. Cons/lazy chains are
	430	+ ## followed link by link; concrete collections are indexed.
	431	+ node: Value
	432	+ backing: seq[Value]
	433	+ idx: int
	434	+ isNode: bool
	435	+
	436	+proc cursor*(v: Value): Cursor =
	437	+ ## Forces nothing: a cons/lazy value is walked link by link, and `hasNext`
	438	+ ## is the only thing that ever forces. So building `(take 3 (map f xs))`
	439	+ ## runs `f` zero times until something asks for an element.
	440	+ if v.isNil: return Cursor(isNode: false)
	441	+ if v.kind in {kCons, kLazy, kNil}: Cursor(isNode: true, node: v)
	442	+ else: Cursor(isNode: false, backing: toSeq(v))
	443	+
	444	+proc hasNext*(c: var Cursor): bool =
	445	+ if c.isNode:
	446	+ c.node = force(c.node)
	447	+ not c.node.isNil and c.node.kind == kCons
	448	+ else: c.idx < c.backing.len
	449	+
	450	+proc next*(c: var Cursor): Value =
	451	+ if c.isNode:
	452	+ result = c.node.head
	453	+ c.node = c.node.tl
	454	+ else:
	455	+ result = c.backing[c.idx]
	456	+ inc c.idx
	457	+
	458	+iterator elems*(v: Value): Value =
	459	+ ## The one way to walk a collection in core: works for lists, vectors, maps,
	460	+ ## sets, strings and lazy seqs alike, and never realizes more than it is asked
	461	+ ## for.
	462	+ var c = cursor(v)
	463	+ while hasNext(c): yield next(c)
	464	+
	465	+proc seqFirst*(v: Value): Value =
	466	+ let f = force(v)
	467	+ if f.isNil: return NilV
	468	+ if f.kind == kCons: return f.head
	469	+ var c = cursor(f)
	470	+ (if hasNext(c): next(c) else: NilV)
	471	+
	472	+proc seqRest*(v: Value): Value =
	473	+ ## The rest of a seq, as a seq. Empty is an empty list, never nil — `next`
	474	+ ## is the one that nils out.
	475	+ let f = force(v)
	476	+ if f.isNil or f.kind == kNil: return mkList(@[])
	477	+ if f.kind == kCons: return (if f.tl.isNil: mkList(@[]) else: f.tl)
	478	+ let xs = toSeq(f)
	479	+ (if xs.len <= 1: mkList(@[]) else: mkList(xs[1 .. ^1]))
	480	+
	481	+proc seqIsEmpty*(v: Value): bool =
	482	+ ## O(1) for lazy seqs: forces at most the first element.
	483	+ var c = cursor(v)
	484	+ not hasNext(c)
	485	+
	486	+proc seqDrop*(v: Value, n: int): Value =
	487	+ ## Skip n elements. Used by `& rest` destructuring, so it must not realize
	488	+ ## anything past the n-th link — `(let [[a b & more] (range)] …)` works.
	489	+ result = v
	490	+ var k = n
	491	+ while k > 0:
	492	+ if seqIsEmpty(result): return mkList(@[])
	493	+ result = seqRest(result)
	494	+ dec k
	495	+
	496	+proc hashValue*(v: Value): uint32 =
	497	+ if v.isNil: return 0
	498	+ case v.kind
	499	+ of kNil: 0'u32
	500	+ of kBool: (if v.b: 0x9e3779b9'u32 else: 0x85ebca6b'u32)
	501	+ of kInt: uint32(hash(v.i))
	502	+ of kFloat:
	503	+ # ints and floats compare equal across kinds, so they must hash alike
	504	+ if v.f == float64(int64(v.f)): uint32(hash(int64(v.f)))
	505	+ else: uint32(hash(v.f))
	506	+ of kStr: mixHash(1'u32, uint32(hash(v.s)))
	507	+ of kKeyword: mixHash(2'u32, uint32(hash(v.s)))
	508	+ of kSymbol: mixHash(3'u32, uint32(hash(v.s)))
	509	+ of kList, kVector, kCons, kLazy:
	510	+ # sequentials are `=` when their elements are, so they hash alike
	511	+ var h = 7'u32
	512	+ for x in elems(v): h = mixHash(h, hashValue(x))
	513	+ h
	514	+ of kSet:
	515	+ var h = 0'u32 # xor: independent of iteration order
	516	+ for e in mapEntries(v.m): h = h xor hashValue(e.key)
	517	+ h
	518	+ of kMap:
	519	+ var h = 0'u32
	520	+ for e in mapEntries(v.m):
	521	+ h = h xor mixHash(hashValue(e.key), hashValue(e.val))
	522	+ h
	523	+ of kFn: uint32(hash(cast[int](cast[pointer](v))))
	524	+
382	525	# --------------------------------------------------------------- accessors
383	526	proc items*(v: Value): seq[Value] =
384	527	## Elements of any sequential value, in order. O(n) — prefer `count`/`nth`
@@ -391,6 +534,11 @@ proc items*(v: Value): seq[Value] =
391	534	var r = newSeqOfCap[Value](v.m.cnt)
392	535	for e in mapEntries(v.m): r.add e.key
393	536	r
	537	+ of kCons, kLazy:
	538	+ var r: seq[Value] = @[]
	539	+ var c = cursor(v)
	540	+ while hasNext(c): r.add next(c)
	541	+ r
394	542	else: @[]
395	543
396	544	proc pairs*(v: Value): seq[(Value, Value)] =
@@ -406,6 +554,12 @@ proc count*(v: Value): int =
406	554	of kVector: v.vec.cnt
407	555	of kMap, kSet: v.m.cnt
408	556	of kStr: v.s.len
	557	+ of kCons, kLazy:
	558	+ # realizes the whole seq, which is the honest cost of counting one
	559	+ var n = 0
	560	+ var c = cursor(v)
	561	+ while hasNext(c): discard next(c); inc n
	562	+ n
409	563	else: err("Don't know how to count: " & prStr(v))
410	564
411	565	proc truthy*(v: Value): bool =
@@ -421,14 +575,16 @@ proc equals*(a, b: Value): bool =
421	575	# numeric tower: int and float compare across types
422	576	if a.kind == kInt and b.kind == kFloat: return float64(a.i) == b.f
423	577	if a.kind == kFloat and b.kind == kInt: return a.f == float64(b.i)
424		- # lists and vectors are sequentially equal in Clojure
425		- if a.kind in {kList, kVector} and b.kind in {kList, kVector}:
426		- if count(a) != count(b): return false
427		- let xs = items(a)
428		- let ys = items(b)
429		- for i in 0 ..< xs.len:
430		- if not equals(xs[i], ys[i]): return false
431		- return true
	578	+ # every sequential thing is `=` to every other with the same elements
	579	+ const Seqs = {kList, kVector, kCons, kLazy}
	580	+ if a.kind in Seqs and b.kind in Seqs:
	581	+ var ca = cursor(a)
	582	+ var cb = cursor(b)
	583	+ while true:
	584	+ let ha = hasNext(ca)
	585	+ if ha != hasNext(cb): return false
	586	+ if not ha: return true
	587	+ if not equals(next(ca), next(cb)): return false
432	588	if a.kind != b.kind: return false
433	589	case a.kind
434	590	of kNil: true
@@ -448,7 +604,7 @@ proc equals*(a, b: Value): bool =
448	604	if not equals(e.val, mapGet(b.m, e.key, missing)): return false
449	605	true
450	606	of kFn: a == b
451		- of kList, kVector: false # handled above
	607	+ of kList, kVector, kCons, kLazy: false # handled above
452	608	# ---------------------------------------------------------------- printing
453	609	proc escapeStr(s: string): string =
454	610	result = "\""
@@ -475,9 +631,10 @@ proc toStr*(v: Value, readable: bool): string =
475	631	of kStr: (if readable: escapeStr(v.s) else: v.s)
476	632	of kKeyword: ":" & v.s
477	633	of kSymbol: v.s
478		- of kList:
	634	+ of kList, kCons, kLazy:
	635	+ # printing a lazy seq realizes it, exactly as in Clojure
479	636	var parts: seq[string] = @[]
480		- for x in v.items: parts.add toStr(x, readable)
	637	+ for x in elems(v): parts.add toStr(x, readable)
481	638	"(" & parts.join(" ") & ")"
482	639	of kVector:
483	640	var parts: seq[string] = @[]
@@ -563,7 +720,7 @@ proc toSeq*(v: Value): seq[Value] =
563	720	if v.isNil: return @[]
564	721	case v.kind
565	722	of kNil: @[]
566		- of kList, kVector, kSet: v.items
	723	+ of kList, kVector, kSet, kCons, kLazy: v.items
567	724	of kStr:
568	725	var r: seq[Value] = @[]
569	726	for c in v.s: r.add mkStr($c)

  type
    Kind* = enum
      kNil, kBool, kInt, kFloat, kStr, kKeyword, kSymbol,
--    kList, kVector, kMap, kSet, kFn
++    kList, kVector, kMap, kSet, kCons, kLazy, kFn
    VNode* = ref object
      ## A trie node: leaves hold values, internal nodes hold children.
      cnt*: int
      nextOrd*: int
--  Value* = ref object
++  Value* = ref ValueObj
++  ValueObj* = object
      case kind*: Kind
      of kNil: discard
      of kBool: b*: bool
      of kList: xs*: seq[Value]
      of kVector: vec*: PVec
      of kMap, kSet: m*: PMap
++    of kCons:
++      head*: Value
++      tl*: Value       ## rest of the seq: a cons, a lazy seq, a coll, or nil
++    of kLazy:
++      thunk*: proc (): Value {.closure.}
++      cached*: Value
++      forced*: bool
      of kFn:
        fn*: proc (args: seq[Value]): Value {.closure.}
        name*: string
    CljError* = object of CatchableError
++# ------------------------------------------------------------ teardown
++## A realized lazy seq is a chain of `cons -> lazy -> cons -> …` refs, and ARC
++## frees a chain by recursing into it — a million-element seq means a million
++## destructor frames, i.e. a segfault at scope exit. So `kCons`/`kLazy` hand
++## their tail to a worklist instead of letting the field drop inline, and the
++## outermost destructor drains it in a loop. Nothing shared is mutated: a node
++## another seq still holds simply survives with its refcount intact.
++##
++## Defining `=destroy` means the compiler stops generating field teardown for
++## `ValueObj`, so every branch below has to release its own fields.
++
++var pendingFree: seq[Value] = @[]
++var draining = false
++
++proc `=destroy`*(x: var ValueObj) =
++  case x.kind
++  of kStr, kKeyword, kSymbol: `=destroy`(x.s)
++  of kList: `=destroy`(x.xs)
++  of kVector: `=destroy`(x.vec)
++  of kMap, kSet: `=destroy`(x.m)
++  of kFn:
++    `=destroy`(x.fn)
++    `=destroy`(x.name)
++  of kCons:
++    `=destroy`(x.head)
++    if not x.tl.isNil: pendingFree.add x.tl   # +1, outlives the release below
++    `=destroy`(x.tl)
++  of kLazy:
++    `=destroy`(x.thunk)
++    if not x.cached.isNil: pendingFree.add x.cached
++    `=destroy`(x.cached)
++  of kNil, kBool, kInt, kFloat: discard
++  if draining: return
++  draining = true
++  while pendingFree.len > 0:
++    let v = pendingFree.pop()
++    discard v      # dies here: its own tail is queued, not recursed into
++  draining = false
++
  let NilV* = Value(kind: kNil)
  let TrueV* = Value(kind: kBool, b: true)
  let FalseV* = Value(kind: kBool, b: false)
  proc equals*(a, b: Value): bool
  proc hashValue*(v: Value): uint32
  proc prStr*(v: Value): string
++proc toSeq*(v: Value): seq[Value]
  # ------------------------------------------------------- persistent vector
  let emptyVNode = VNode(leaf: false, kids: @[])
    collect(m.root, result)
    result.sort(proc (a, b: MEntry): int = cmp(a.ord, b.ord))
--proc hashValue*(v: Value): uint32 =
--  if v.isNil: return 0
--  case v.kind
--  of kNil: 0'u32
--  of kBool: (if v.b: 0x9e3779b9'u32 else: 0x85ebca6b'u32)
--  of kInt: uint32(hash(v.i))
--  of kFloat:
--    # ints and floats compare equal across kinds, so they must hash alike
--    if v.f == float64(int64(v.f)): uint32(hash(int64(v.f)))
--    else: uint32(hash(v.f))
--  of kStr: mixHash(1'u32, uint32(hash(v.s)))
--  of kKeyword: mixHash(2'u32, uint32(hash(v.s)))
--  of kSymbol: mixHash(3'u32, uint32(hash(v.s)))
--  of kList, kVector:
--    # lists and vectors are `=` when their elements are, so they hash alike
--    var h = 7'u32
--    for x in (if v.kind == kList: v.xs else: vecToSeq(v.vec)):
--      h = mixHash(h, hashValue(x))
--    h
--  of kSet:
--    var h = 0'u32                       # xor: independent of iteration order
--    for e in mapEntries(v.m): h = h xor hashValue(e.key)
--    h
--  of kMap:
--    var h = 0'u32
--    for e in mapEntries(v.m):
--      h = h xor mixHash(hashValue(e.key), hashValue(e.val))
--    h
--  of kFn: uint32(hash(cast[int](cast[pointer](v))))
--
  # ------------------------------------------------------------ constructors
  proc mkBool*(x: bool): Value = (if x: TrueV else: FalseV)
  proc mkInt*(x: int64): Value = Value(kind: kInt, i: x)
  proc mkFn*(name: string, f: proc (args: seq[Value]): Value {.closure.}): Value =
    Value(kind: kFn, fn: f, name: name)
++# --------------------------------------------------------------- lazy seqs
++## A lazy seq is a thunk that, when forced, yields either nil/`kNil` (the end)
++## or a cons cell whose tail is usually another lazy seq. Forcing is memoized
++## in place, so each element is computed once no matter how often it is walked.
++## Nothing here recurses per element: `force` loops, and so does every producer
++## in core, which is what keeps `(nth (iterate inc 0) 1000000)` from blowing the
++## stack.
++
++proc mkCons*(h, t: Value): Value = Value(kind: kCons, head: h, tl: t)
++
++proc mkLazy*(f: proc (): Value {.closure.}): Value =
++  Value(kind: kLazy, thunk: f, cached: nil, forced: false)
++
++proc force*(v: Value): Value =
++  ## Realize one step: follow a chain of lazy seqs down to a cons, a concrete
++  ## collection, or the end of the seq.
++  var cur = v
++  while not cur.isNil and cur.kind == kLazy:
++    if not cur.forced:
++      cur.cached = cur.thunk()
++      cur.forced = true
++      cur.thunk = nil       # drop the closure so its captures can be collected
++    cur = cur.cached
++  cur
++
++proc isSeqNode(v: Value): bool =
++  not v.isNil and v.kind in {kCons, kLazy}
++
++type Cursor* = object
++  ## Walks any seqable value without materializing it. Cons/lazy chains are
++  ## followed link by link; concrete collections are indexed.
++  node: Value
++  backing: seq[Value]
++  idx: int
++  isNode: bool
++
++proc cursor*(v: Value): Cursor =
++  ## Forces nothing: a cons/lazy value is walked link by link, and `hasNext`
++  ## is the only thing that ever forces. So building `(take 3 (map f xs))`
++  ## runs `f` zero times until something asks for an element.
++  if v.isNil: return Cursor(isNode: false)
++  if v.kind in {kCons, kLazy, kNil}: Cursor(isNode: true, node: v)
++  else: Cursor(isNode: false, backing: toSeq(v))
++
++proc hasNext*(c: var Cursor): bool =
++  if c.isNode:
++    c.node = force(c.node)
++    not c.node.isNil and c.node.kind == kCons
++  else: c.idx < c.backing.len
++
++proc next*(c: var Cursor): Value =
++  if c.isNode:
++    result = c.node.head
++    c.node = c.node.tl
++  else:
++    result = c.backing[c.idx]
++    inc c.idx
++
++iterator elems*(v: Value): Value =
++  ## The one way to walk a collection in core: works for lists, vectors, maps,
++  ## sets, strings and lazy seqs alike, and never realizes more than it is asked
++  ## for.
++  var c = cursor(v)
++  while hasNext(c): yield next(c)
++
++proc seqFirst*(v: Value): Value =
++  let f = force(v)
++  if f.isNil: return NilV
++  if f.kind == kCons: return f.head
++  var c = cursor(f)
++  (if hasNext(c): next(c) else: NilV)
++
++proc seqRest*(v: Value): Value =
++  ## The rest of a seq, as a seq. Empty is an empty list, never nil — `next`
++  ## is the one that nils out.
++  let f = force(v)
++  if f.isNil or f.kind == kNil: return mkList(@[])
++  if f.kind == kCons: return (if f.tl.isNil: mkList(@[]) else: f.tl)
++  let xs = toSeq(f)
++  (if xs.len <= 1: mkList(@[]) else: mkList(xs[1 .. ^1]))
++
++proc seqIsEmpty*(v: Value): bool =
++  ## O(1) for lazy seqs: forces at most the first element.
++  var c = cursor(v)
++  not hasNext(c)
++
++proc seqDrop*(v: Value, n: int): Value =
++  ## Skip n elements. Used by `& rest` destructuring, so it must not realize
++  ## anything past the n-th link — `(let [[a b & more] (range)] …)` works.
++  result = v
++  var k = n
++  while k > 0:
++    if seqIsEmpty(result): return mkList(@[])
++    result = seqRest(result)
++    dec k
++
++proc hashValue*(v: Value): uint32 =
++  if v.isNil: return 0
++  case v.kind
++  of kNil: 0'u32
++  of kBool: (if v.b: 0x9e3779b9'u32 else: 0x85ebca6b'u32)
++  of kInt: uint32(hash(v.i))
++  of kFloat:
++    # ints and floats compare equal across kinds, so they must hash alike
++    if v.f == float64(int64(v.f)): uint32(hash(int64(v.f)))
++    else: uint32(hash(v.f))
++  of kStr: mixHash(1'u32, uint32(hash(v.s)))
++  of kKeyword: mixHash(2'u32, uint32(hash(v.s)))
++  of kSymbol: mixHash(3'u32, uint32(hash(v.s)))
++  of kList, kVector, kCons, kLazy:
++    # sequentials are `=` when their elements are, so they hash alike
++    var h = 7'u32
++    for x in elems(v): h = mixHash(h, hashValue(x))
++    h
++  of kSet:
++    var h = 0'u32                       # xor: independent of iteration order
++    for e in mapEntries(v.m): h = h xor hashValue(e.key)
++    h
++  of kMap:
++    var h = 0'u32
++    for e in mapEntries(v.m):
++      h = h xor mixHash(hashValue(e.key), hashValue(e.val))
++    h
++  of kFn: uint32(hash(cast[int](cast[pointer](v))))
++
  # --------------------------------------------------------------- accessors
  proc items*(v: Value): seq[Value] =
    ## Elements of any sequential value, in order. O(n) — prefer `count`/`nth`
      var r = newSeqOfCap[Value](v.m.cnt)
      for e in mapEntries(v.m): r.add e.key
      r
++  of kCons, kLazy:
++    var r: seq[Value] = @[]
++    var c = cursor(v)
++    while hasNext(c): r.add next(c)
++    r
    else: @[]
  proc pairs*(v: Value): seq[(Value, Value)] =
    of kVector: v.vec.cnt
    of kMap, kSet: v.m.cnt
    of kStr: v.s.len
++  of kCons, kLazy:
++    # realizes the whole seq, which is the honest cost of counting one
++    var n = 0
++    var c = cursor(v)
++    while hasNext(c): discard next(c); inc n
++    n
    else: err("Don't know how to count: " & prStr(v))
  proc truthy*(v: Value): bool =
    # numeric tower: int and float compare across types
    if a.kind == kInt and b.kind == kFloat: return float64(a.i) == b.f
    if a.kind == kFloat and b.kind == kInt: return a.f == float64(b.i)
--  # lists and vectors are sequentially equal in Clojure
++  # every sequential thing is `=` to every other with the same elements
--  if a.kind in {kList, kVector} and b.kind in {kList, kVector}:
++  const Seqs = {kList, kVector, kCons, kLazy}
--    if count(a) != count(b): return false
++  if a.kind in Seqs and b.kind in Seqs:
--    let xs = items(a)
++    var ca = cursor(a)
--    let ys = items(b)
++    var cb = cursor(b)
--    for i in 0 ..< xs.len:
++    while true:
--      if not equals(xs[i], ys[i]): return false
++      let ha = hasNext(ca)
--    return true
++      if ha != hasNext(cb): return false
++      if not ha: return true
++      if not equals(next(ca), next(cb)): return false
    if a.kind != b.kind: return false
    case a.kind
    of kNil: true
        if not equals(e.val, mapGet(b.m, e.key, missing)): return false
      true
    of kFn: a == b
--  of kList, kVector: false  # handled above
++  of kList, kVector, kCons, kLazy: false  # handled above
  # ---------------------------------------------------------------- printing
  proc escapeStr(s: string): string =
    result = "\""
    of kStr: (if readable: escapeStr(v.s) else: v.s)
    of kKeyword: ":" & v.s
    of kSymbol: v.s
--  of kList:
++  of kList, kCons, kLazy:
++    # printing a lazy seq realizes it, exactly as in Clojure
      var parts: seq[string] = @[]
--    for x in v.items: parts.add toStr(x, readable)
++    for x in elems(v): parts.add toStr(x, readable)
      "(" & parts.join(" ") & ")"
    of kVector:
      var parts: seq[string] = @[]
    if v.isNil: return @[]
    case v.kind
    of kNil: @[]
--  of kList, kVector, kSet: v.items
++  of kList, kVector, kSet, kCons, kLazy: v.items
    of kStr:
      var r: seq[Value] = @[]
      for c in v.s: r.add mkStr($c)

added tests/lazy.expected +24 -0 Viewed

new file mode 100644
@@ -0,0 +1,24 @@
	1	+(0 1 2 3 4)
	2	+(1 2 3 4 5)
	3	+(0 2 4 6 8)
	4	+(1 2 4 8 16)
	5	+(:x :x :x :x) (:y :y :y)
	6	+(1 2 3 1 2 3 1)
	7	+(100 101 102)
	8	+(0 1 2 3 4)
	9	+(10 11 12)
	10	+(1 2 0 1 2)
	11	+1 1 1000
	12	+built, calls so far: 0
	13	+took (0 1 2) - calls: 3
	14	+took (0 1 2) - calls: 3 (memoized)
	15	+1330
	16	+0 1 (2 3 4)
	17	+(2 3 4) (2 4)
	18	+(0 1 2 3 4) (2 4 6) (5 4 3 2 1)
	19	+100 true nil (0 1)
	20	+true true
	21	+[0 1 2] (6 7 8 9)
	22	+(0 0 1 2) (1 2) nil 3
	23	+1 2 3
	24	+200000 200000

++(0 1 2 3 4)
++(1 2 3 4 5)
++(0 2 4 6 8)
++(1 2 4 8 16)
++(:x :x :x :x) (:y :y :y)
++(1 2 3 1 2 3 1)
++(100 101 102)
++(0 1 2 3 4)
++(10 11 12)
++(1 2 0 1 2)
++1 1 1000
++built, calls so far: 0
++took (0 1 2) - calls: 3
++took (0 1 2) - calls: 3 (memoized)
++1330
++0 1 (2 3 4)
++(2 3 4) (2 4)
++(0 1 2 3 4) (2 4 6) (5 4 3 2 1)
++100 true nil (0 1)
++true true
++[0 1 2] (6 7 8 9)
++(0 0 1 2) (1 2) nil 3
++1 2 3
++200000 200000