Lower `-> impl Iterator` to a Nim closure iterator
Return-position `impl Trait` is an opaque type the caller cannot name, so it had been rejected wholesale. But `impl Iterator` is 354 of the 690 occurrences, and Nim's closure iterator is a real equivalent -- lazy, first-class, resumable -- rather than an approximation. 59 blockers down to 29. Laziness is the whole point of not materialising a seq, so it is checked rather than asserted: iterating (0..1_000_000).map(..) and breaking after three elements returns in 3ms. This also revises an earlier judgement. `.map` and `.filter` had been rejected because "silently dropping an adaptor would change which elements the loop visits" -- but implementing one drops nothing. In a lazy iterator map is a transform of the yielded value and filter is a continue, and no element is computed that Rust would not compute. They are supported now. Three things this needed, each found by running it rather than by reasoning: `for x in f()()` never terminates. Nim re-evaluates the call on every step, building a fresh iterator each time, so it prints one element forever. The iterator has to be bound before it is iterated. Nim refuses to capture an openArray -- "cannot be captured as it would violate memory safety" -- which is Nim enforcing the property Rust states with `+ '_`. Each captured view is copied once per call into a seq the closure can own. Rust guarantees the borrow outlives the iterator, so nothing can mutate the original while it is read, which makes the copy a cost rather than a semantic change. It is still a copy where Rust has none, and DESIGN.md says so. Two adaptors in one chain may name their closure parameters the same, as `.filter(|x| ..).map(|x| ..)` does. The element is bound once under a fresh name and each parameter aliases it, since two `let x` collide in one Nim scope. Survey unchanged at 18 of 400 accepted, 12 Nim-compilable. Tenth feature with that result. 47 differential cases, all green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fe27b4b parent: 4e4d09d modified
DESIGN.md +43 -3 | @@ -736,9 +736,49 @@ is `fn f<A: T>(x: A)`. Bounds are already dropped (see Generics), so what is | ||
| 736 | 736 | left is a fresh parameter, and Nim instantiates it structurally at the call |
| 737 | 737 | site exactly as Rust does. That is now what happens — 86 blockers down to 59. |
| 738 | 738 | |
| 739 | -Return position is genuinely different: the caller cannot name the type, and | |
| 740 | -Nim has no equivalent. It is still rejected, but the message now says which | |
| 741 | -trait and that argument position would have been fine. | |
| 739 | +Return position is genuinely different: the caller cannot name the type. But | |
| 740 | +`impl Iterator` is 354 of the 690 occurrences, and **Nim's closure iterator is | |
| 741 | +a real equivalent** — lazy, first-class, resumable — so that case is lowered | |
| 742 | +rather than rejected. 59 blockers down to 29. | |
| 743 | + | |
| 744 | +```nim | |
| 745 | +proc doubled*(xs: openArray[int32]): iterator (): int32 {.closure.} = | |
| 746 | + let rsTmpCap1: seq[int32] = @xs | |
| 747 | + result = iterator (): int32 {.closure.} = | |
| 748 | + for i in 0 ..< int(rsTmpCap1.len): | |
| 749 | + yield rsTmpCap1[i] * 2'i32 | |
| 750 | +``` | |
| 751 | + | |
| 752 | +Laziness is the point and is checked, not assumed: iterating | |
| 753 | +`(0..1_000_000).map(..)` and breaking after three elements returns in 3ms. A | |
| 754 | +materialised `seq` could not do that, which is why this is not "collect into a | |
| 755 | +sequence and iterate that". | |
| 756 | + | |
| 757 | +`.map` and `.filter` are supported here, and this revises an earlier | |
| 758 | +judgement. They had been rejected on the grounds that "silently dropping an | |
| 759 | +adaptor would change which elements the loop visits" — but *implementing* one | |
| 760 | +drops nothing: in a lazy iterator `map` is a transform of the yielded value | |
| 761 | +and `filter` is a `continue`, and no element is computed that Rust would not | |
| 762 | +compute. | |
| 763 | + | |
| 764 | +Three things this needed that are worth recording: | |
| 765 | + | |
| 766 | +- **`for x in f()()` never terminates.** Nim re-evaluates the call on every | |
| 767 | + step, building a fresh iterator each time. The iterator must be bound first. | |
| 768 | + Found by writing exactly that and watching it print one element forever. | |
| 769 | +- **Nim refuses to capture an `openArray`** — "cannot be captured as it would | |
| 770 | + violate memory safety". That is Nim enforcing the property Rust states with | |
| 771 | + `+ '_`. Each captured view is copied once per call into a `seq` the closure | |
| 772 | + can own. Rust guarantees the borrow outlives the iterator, so nothing can | |
| 773 | + mutate the original while it is read: the copy is a cost, not a semantic | |
| 774 | + change. It is still a copy where Rust has none. | |
| 775 | +- Two adaptors in one chain may name their closure parameters the same | |
| 776 | + (`xs.iter().filter(|x| ..).map(|x| ..)`). The element is bound once under a | |
| 777 | + fresh name and each parameter aliases it, since two `let x` would collide in | |
| 778 | + one Nim scope. | |
| 779 | + | |
| 780 | +What remains in return position is `impl Future` (async), `impl Display`, and | |
| 781 | +traits from other crates — those are still rejected, with the trait named. | |
| 742 | 782 | |
| 743 | 783 | ### Trait methods are named by trait, not by type |
| 744 | 784 | |
| @@ -736,9 +736,49 @@ is `fn f<A: T>(x: A)`. Bounds are already dropped (see Generics), so what is | |||
| 736 | left is a fresh parameter, and Nim instantiates it structurally at the call | 736 | left is a fresh parameter, and Nim instantiates it structurally at the call |
| 737 | site exactly as Rust does. That is now what happens — 86 blockers down to 59. | 737 | site exactly as Rust does. That is now what happens — 86 blockers down to 59. |
| 738 | 738 | ||
| 739 | -Return position is genuinely different: the caller cannot name the type, and | 739 | +Return position is genuinely different: the caller cannot name the type. But |
| 740 | -Nim has no equivalent. It is still rejected, but the message now says which | 740 | +`impl Iterator` is 354 of the 690 occurrences, and **Nim's closure iterator is |
| 741 | -trait and that argument position would have been fine. | 741 | +a real equivalent** — lazy, first-class, resumable — so that case is lowered |
| 742 | +rather than rejected. 59 blockers down to 29. | ||
| 743 | + | ||
| 744 | +```nim | ||
| 745 | +proc doubled*(xs: openArray[int32]): iterator (): int32 {.closure.} = | ||
| 746 | + let rsTmpCap1: seq[int32] = @xs | ||
| 747 | + result = iterator (): int32 {.closure.} = | ||
| 748 | + for i in 0 ..< int(rsTmpCap1.len): | ||
| 749 | + yield rsTmpCap1[i] * 2'i32 | ||
| 750 | +``` | ||
| 751 | + | ||
| 752 | +Laziness is the point and is checked, not assumed: iterating | ||
| 753 | +`(0..1_000_000).map(..)` and breaking after three elements returns in 3ms. A | ||
| 754 | +materialised `seq` could not do that, which is why this is not "collect into a | ||
| 755 | +sequence and iterate that". | ||
| 756 | + | ||
| 757 | +`.map` and `.filter` are supported here, and this revises an earlier | ||
| 758 | +judgement. They had been rejected on the grounds that "silently dropping an | ||
| 759 | +adaptor would change which elements the loop visits" — but *implementing* one | ||
| 760 | +drops nothing: in a lazy iterator `map` is a transform of the yielded value | ||
| 761 | +and `filter` is a `continue`, and no element is computed that Rust would not | ||
| 762 | +compute. | ||
| 763 | + | ||
| 764 | +Three things this needed that are worth recording: | ||
| 765 | + | ||
| 766 | +- **`for x in f()()` never terminates.** Nim re-evaluates the call on every | ||
| 767 | + step, building a fresh iterator each time. The iterator must be bound first. | ||
| 768 | + Found by writing exactly that and watching it print one element forever. | ||
| 769 | +- **Nim refuses to capture an `openArray`** — "cannot be captured as it would | ||
| 770 | + violate memory safety". That is Nim enforcing the property Rust states with | ||
| 771 | + `+ '_`. Each captured view is copied once per call into a `seq` the closure | ||
| 772 | + can own. Rust guarantees the borrow outlives the iterator, so nothing can | ||
| 773 | + mutate the original while it is read: the copy is a cost, not a semantic | ||
| 774 | + change. It is still a copy where Rust has none. | ||
| 775 | +- Two adaptors in one chain may name their closure parameters the same | ||
| 776 | + (`xs.iter().filter(|x| ..).map(|x| ..)`). The element is bound once under a | ||
| 777 | + fresh name and each parameter aliases it, since two `let x` would collide in | ||
| 778 | + one Nim scope. | ||
| 779 | + | ||
| 780 | +What remains in return position is `impl Future` (async), `impl Display`, and | ||
| 781 | +traits from other crates — those are still rejected, with the trait named. | ||
| 742 | 782 | ||
| 743 | ### Trait methods are named by trait, not by type | 783 | ### Trait methods are named by trait, not by type |
| 744 | 784 | ||
modified
README.md +8 -0 | @@ -120,6 +120,14 @@ produce Nim the Nim compiler accepts. (Accepted, compiles, and | ||
| 120 | 120 | behaviourally verified are three different bars; only `tests/cases/` clears |
| 121 | 121 | the third.) |
| 122 | 122 | |
| 123 | +## Iterators | |
| 124 | + | |
| 125 | +`-> impl Iterator<Item = T>` becomes a Nim closure iterator — lazy, | |
| 126 | +first-class, resumable, which is what Rust's is. Breaking out of a | |
| 127 | +million-element iterator after three elements returns in 3ms, so this is not | |
| 128 | +"collect into a sequence". `.map` and `.filter` work in that form, where they | |
| 129 | +are exactly a transform and a skip. | |
| 130 | + | |
| 123 | 131 | ## Does it generalise? |
| 124 | 132 | |
| 125 | 133 | `base16ct` is the crate this was built toward, so a second one was tried. |
| @@ -120,6 +120,14 @@ produce Nim the Nim compiler accepts. (Accepted, compiles, and | |||
| 120 | behaviourally verified are three different bars; only `tests/cases/` clears | 120 | behaviourally verified are three different bars; only `tests/cases/` clears |
| 121 | the third.) | 121 | the third.) |
| 122 | 122 | ||
| 123 | +## Iterators | ||
| 124 | + | ||
| 125 | +`-> impl Iterator<Item = T>` becomes a Nim closure iterator — lazy, | ||
| 126 | +first-class, resumable, which is what Rust's is. Breaking out of a | ||
| 127 | +million-element iterator after three elements returns in 3ms, so this is not | ||
| 128 | +"collect into a sequence". `.map` and `.filter` work in that form, where they | ||
| 129 | +are exactly a transform and a skip. | ||
| 130 | + | ||
| 123 | ## Does it generalise? | 131 | ## Does it generalise? |
| 124 | 132 | ||
| 125 | `base16ct` is the crate this was built toward, so a second one was tried. | 133 | `base16ct` is the crate this was built toward, so a second one was tried. |
modified
src/lower.rs +268 -3 | @@ -83,6 +83,15 @@ enum Iter { | ||
| 83 | 83 | Enumerate(Box<Iter>), |
| 84 | 84 | /// `.zip(other)` — stops at the shorter, as Rust's does. |
| 85 | 85 | Zip(Box<Iter>, Box<Iter>), |
| 86 | + /// `.map(|x| ..)`: the element is the closure's body, with its parameter | |
| 87 | + /// bound to the incoming element. | |
| 88 | + Map { inner: Box<Iter>, param: String, body: Box<Expr> }, | |
| 89 | + /// A Nim closure iterator value, iterated directly rather than by index. | |
| 90 | + Closure { code: String, elem: Option<Nim> }, | |
| 91 | + /// `.filter(|x| ..)`: elements the predicate rejects are skipped. In a | |
| 92 | + /// lazy iterator this is exact; it is only in an index loop that it | |
| 93 | + /// changes the loop's shape. | |
| 94 | + Filter { inner: Box<Iter>, param: String, body: Box<Expr> }, | |
| 86 | 95 | } |
| 87 | 96 | |
| 88 | 97 | impl Iter { |
| @@ -98,6 +107,9 @@ impl Iter { | ||
| 98 | 107 | Iter::Chunks { k, len, .. } => format!("({} div int({}))", len, k), |
| 99 | 108 | Iter::Windows { len, k, .. } => format!("(max(0, {} - int({}) + 1))", len, k), |
| 100 | 109 | Iter::Enumerate(i) => i.len(), |
| 110 | + Iter::Map { inner, .. } | Iter::Filter { inner, .. } => inner.len(), | |
| 111 | + // A closure iterator has no length; it is iterated directly. | |
| 112 | + Iter::Closure { .. } => "0".into(), | |
| 101 | 113 | Iter::Zip(a, b) => format!("min({}, {})", a.len(), b.len()), |
| 102 | 114 | } |
| 103 | 115 | } |
| @@ -2171,6 +2183,31 @@ impl Lowerer { | ||
| 2171 | 2183 | } else { |
| 2172 | 2184 | self.target.replace(("result".to_string(), Some(ret.clone()))) |
| 2173 | 2185 | }; |
| 2186 | + // `-> impl Iterator<Item = T>` becomes a Nim closure iterator, which | |
| 2187 | + // is lazy and first-class as Rust's is. The body's trailing expression | |
| 2188 | + // is the chain it yields from. | |
| 2189 | + if let Nim::Iter(elem) = &ret { | |
| 2190 | + let elem = (**elem).clone(); | |
| 2191 | + // The *declared* parameter types, not the scope's, which record | |
| 2192 | + // the owned form. | |
| 2193 | + let views: Vec<(String, Nim)> = typed | |
| 2194 | + .iter() | |
| 2195 | + .zip(ptys.iter()) | |
| 2196 | + .filter(|(_, t)| matches!(t, Nim::OpenArray(_))) | |
| 2197 | + .filter_map(|(p, t)| match &*p.pat { | |
| 2198 | + Pat::Ident(i) => Some((i.ident.to_string(), t.clone())), | |
| 2199 | + _ => None, | |
| 2200 | + }) | |
| 2201 | + .collect(); | |
| 2202 | + let r = self.iter_body(body, &elem, &views); | |
| 2203 | + self.indent -= 1; | |
| 2204 | + self.ret = outer_ret; | |
| 2205 | + self.fn_generics = outer_fg; | |
| 2206 | + self.target = outer_target; | |
| 2207 | + self.pop_scope(); | |
| 2208 | + self.blank(); | |
| 2209 | + return r; | |
| 2210 | + } | |
| 2174 | 2211 | let before = self.out.len(); |
| 2175 | 2212 | let tail = self.block_body_at(body, Some(&ret))?; |
| 2176 | 2213 | self.target = outer_target; |
| @@ -2687,6 +2724,34 @@ impl Lowerer { | ||
| 2687 | 2724 | } |
| 2688 | 2725 | let it = self.resolve_iter(&f.expr)?; |
| 2689 | 2726 | |
| 2727 | + // A closure iterator is iterated directly; there is no index. | |
| 2728 | + if let Iter::Closure { code, elem } = &it { | |
| 2729 | + let name = match &*f.pat { | |
| 2730 | + Pat::Ident(i) => i.ident.to_string(), | |
| 2731 | + Pat::Wild(_) => "_".into(), | |
| 2732 | + _ => return Err("unsupported `for` pattern over an iterator".into()), | |
| 2733 | + }; | |
| 2734 | + self.line(&format!("for {} in {}():", ident(&name), code)); | |
| 2735 | + self.indent += 1; | |
| 2736 | + self.push_scope(); | |
| 2737 | + if let Some(t) = elem { | |
| 2738 | + self.bind(&name, t.clone()); | |
| 2739 | + } | |
| 2740 | + let before = self.out.len(); | |
| 2741 | + let saved = self.target.take(); | |
| 2742 | + if let Some(v) = self.block_body(&f.body)? { | |
| 2743 | + let code = v.code.clone(); | |
| 2744 | + self.line(&format!("discard {code}")); | |
| 2745 | + } | |
| 2746 | + self.target = saved; | |
| 2747 | + if self.out.len() == before { | |
| 2748 | + self.line("discard"); | |
| 2749 | + } | |
| 2750 | + self.pop_scope(); | |
| 2751 | + self.indent -= 1; | |
| 2752 | + return Ok(()); | |
| 2753 | + } | |
| 2754 | + | |
| 2690 | 2755 | // One index loop drives the whole chain. Rust's adaptors are lazy and |
| 2691 | 2756 | // compose; resolving them to an index and binding each name to an |
| 2692 | 2757 | // lvalue reproduces that without materialising anything. |
| @@ -2753,6 +2818,39 @@ impl Lowerer { | ||
| 2753 | 2818 | } |
| 2754 | 2819 | Ok(it) |
| 2755 | 2820 | } |
| 2821 | + "map" | "filter" if m.args.len() == 1 => { | |
| 2822 | + let Expr::Closure(c) = &m.args[0] else { | |
| 2823 | + return Err(format!( | |
| 2824 | + "`.{name}()` needs a closure written at the call \ | |
| 2825 | + site; a function value would need its element \ | |
| 2826 | + type, which is not tracked here" | |
| 2827 | + )); | |
| 2828 | + }; | |
| 2829 | + if c.inputs.len() != 1 { | |
| 2830 | + return Err(format!("`.{name}()` takes a one-argument closure")); | |
| 2831 | + } | |
| 2832 | + let param = match &c.inputs[0] { | |
| 2833 | + Pat::Ident(i) => i.ident.to_string(), | |
| 2834 | + Pat::Wild(_) => "unused0".into(), | |
| 2835 | + Pat::Reference(r) => match &*r.pat { | |
| 2836 | + Pat::Ident(i) => i.ident.to_string(), | |
| 2837 | + _ => return Err("unsupported closure parameter".into()), | |
| 2838 | + }, | |
| 2839 | + _ => return Err("unsupported closure parameter".into()), | |
| 2840 | + }; | |
| 2841 | + let body = match &*c.body { | |
| 2842 | + Expr::Block(b) => single_expr(&b.block) | |
| 2843 | + .ok_or("a closure body with statements is not implemented yet")? | |
| 2844 | + .clone(), | |
| 2845 | + other => other.clone(), | |
| 2846 | + }; | |
| 2847 | + let inner = Box::new(self.resolve_iter(&m.receiver)?); | |
| 2848 | + Ok(if name == "map" { | |
| 2849 | + Iter::Map { inner, param, body: Box::new(body) } | |
| 2850 | + } else { | |
| 2851 | + Iter::Filter { inner, param, body: Box::new(body) } | |
| 2852 | + }) | |
| 2853 | + } | |
| 2756 | 2854 | "enumerate" if m.args.is_empty() => { |
| 2757 | 2855 | Ok(Iter::Enumerate(Box::new(self.resolve_iter(&m.receiver)?))) |
| 2758 | 2856 | } |
| @@ -2790,6 +2888,14 @@ impl Lowerer { | ||
| 2790 | 2888 | if let Some(Alias::Iterator(it)) = self.lookup_alias(&n) { |
| 2791 | 2889 | return Ok((*it).clone()); |
| 2792 | 2890 | } |
| 2891 | + // A binding whose type *is* an iterator -- a parameter of | |
| 2892 | + // `impl Iterator` type, say -- is iterated directly. | |
| 2893 | + if let Some(Nim::Iter(elem)) = self.lookup(&n) { | |
| 2894 | + return Ok(Iter::Closure { | |
| 2895 | + code: ident(&n), | |
| 2896 | + elem: Some((*elem).clone()), | |
| 2897 | + }); | |
| 2898 | + } | |
| 2793 | 2899 | if let Some(Alias::Window { code, off, len, elem }) = self.lookup_alias(&n) { |
| 2794 | 2900 | return Ok(Iter::Elems { code, off, len, elem, mutable: false }); |
| 2795 | 2901 | } |
| @@ -2802,6 +2908,17 @@ impl Lowerer { | ||
| 2802 | 2908 | mutable: false, |
| 2803 | 2909 | }) |
| 2804 | 2910 | } |
| 2911 | + other if matches!(self.iter_value_type(other), Some(_)) => { | |
| 2912 | + // A value that *is* an iterator -- typically a call returning | |
| 2913 | + // `impl Iterator`. It must be bound before being iterated: | |
| 2914 | + // `for x in f()()` would rebuild it on every step and never | |
| 2915 | + // terminate. Probed against Nim 2.2.4. | |
| 2916 | + let elem = self.iter_value_type(other).unwrap(); | |
| 2917 | + let v = self.expr(other)?; | |
| 2918 | + let tmp = self.fresh("It"); | |
| 2919 | + self.line(&format!("let {} = {}", tmp, v.code)); | |
| 2920 | + Ok(Iter::Closure { code: tmp, elem: Some(elem) }) | |
| 2921 | + } | |
| 2805 | 2922 | other => { |
| 2806 | 2923 | // A `for` binding that is itself a window iterates that window, |
| 2807 | 2924 | // not the whole container it points into. |
| @@ -2820,6 +2937,93 @@ impl Lowerer { | ||
| 2820 | 2937 | } |
| 2821 | 2938 | } |
| 2822 | 2939 | |
| 2940 | + /// The body of a function returning `impl Iterator`: everything but the | |
| 2941 | + /// trailing expression runs once, then the chain becomes a closure | |
| 2942 | + /// iterator that yields. | |
| 2943 | + fn iter_body( | |
| 2944 | + &mut self, | |
| 2945 | + body: &syn::Block, | |
| 2946 | + elem: &Nim, | |
| 2947 | + views: &[(String, Nim)], | |
| 2948 | + ) -> Result<(), String> { | |
| 2949 | + let n = body.stmts.len(); | |
| 2950 | + let Some(Stmt::Expr(last, None)) = body.stmts.last() else { | |
| 2951 | + return Err("a function returning `impl Iterator` must end in the \ | |
| 2952 | + iterator expression" | |
| 2953 | + .into()); | |
| 2954 | + }; | |
| 2955 | + for st in &body.stmts[..n - 1] { | |
| 2956 | + self.stmt(st)?; | |
| 2957 | + } | |
| 2958 | + // Nim will not let a closure capture an `openArray`: it is a view, | |
| 2959 | + // and capturing it could outlive what it points at. That is the same | |
| 2960 | + // property Rust states with `+ '_`. Each such parameter is copied once | |
| 2961 | + // per call into a `seq` the closure can own. Rust guarantees the | |
| 2962 | + // borrow outlives the iterator, so nothing can mutate the original | |
| 2963 | + // while it is being read, which is what makes the copy unobservable -- | |
| 2964 | + // it is a cost, not a semantic change. | |
| 2965 | + for (n, t) in views { | |
| 2966 | + let copy = self.fresh("Cap"); | |
| 2967 | + let elem_t = match t { | |
| 2968 | + Nim::OpenArray(e) => (**e).clone(), | |
| 2969 | + other => other.clone(), | |
| 2970 | + }; | |
| 2971 | + self.line(&format!("let {}: seq[{}] = @{}", copy, elem_t.render(), ident(n))); | |
| 2972 | + self.bind_alias( | |
| 2973 | + n, | |
| 2974 | + Alias::Value { code: copy, ty: Some(Nim::Seq(Box::new(elem_t))) }, | |
| 2975 | + ); | |
| 2976 | + } | |
| 2977 | + | |
| 2978 | + let it = self.resolve_iter(last)?; | |
| 2979 | + self.line(&format!( | |
| 2980 | + "result = iterator (): {} {{.closure.}} =", | |
| 2981 | + elem.render() | |
| 2982 | + )); | |
| 2983 | + self.indent += 1; | |
| 2984 | + let i = self.fresh("Idx"); | |
| 2985 | + self.line(&format!("for {} in 0 ..< int({}):", i, it.len())); | |
| 2986 | + self.indent += 1; | |
| 2987 | + self.push_scope(); | |
| 2988 | + let name = self.fresh("Elem"); | |
| 2989 | + self.bind_pattern_named(&name, &it, &i)?; | |
| 2990 | + // The binding may be an alias rather than a Nim variable, so the | |
| 2991 | + // yielded expression is looked up rather than assumed to be the name. | |
| 2992 | + let v = match self.lookup_alias(&name) { | |
| 2993 | + Some(Alias::Value { code, .. }) => code, | |
| 2994 | + _ => ident(&name), | |
| 2995 | + }; | |
| 2996 | + self.line(&format!("yield {}", v)); | |
| 2997 | + self.pop_scope(); | |
| 2998 | + self.indent -= 2; | |
| 2999 | + Ok(()) | |
| 3000 | + } | |
| 3001 | + | |
| 3002 | + /// Bind one element of an iterator under a fresh name, and return how to | |
| 3003 | + /// refer to it and what type it has. | |
| 3004 | + fn bind_element(&mut self, it: &Iter, i: &str) -> Result<(String, Option<Nim>), String> { | |
| 3005 | + let n = self.fresh("El"); | |
| 3006 | + self.bind_pattern_named(&n, it, i)?; | |
| 3007 | + Ok(match self.lookup_alias(&n) { | |
| 3008 | + Some(Alias::Value { code, ty }) => (code, ty), | |
| 3009 | + _ => (ident(&n), self.lookup(&n)), | |
| 3010 | + }) | |
| 3011 | + } | |
| 3012 | + | |
| 3013 | + /// Bind one name against a resolved iterator, for a closure parameter. | |
| 3014 | + fn bind_pattern_named(&mut self, name: &str, it: &Iter, i: &str) -> Result<(), String> { | |
| 3015 | + let id: syn::Ident = | |
| 3016 | + syn::parse_str(name).map_err(|e: syn::Error| e.to_string())?; | |
| 3017 | + let pat = Pat::Ident(syn::PatIdent { | |
| 3018 | + attrs: Vec::new(), | |
| 3019 | + by_ref: None, | |
| 3020 | + mutability: None, | |
| 3021 | + ident: id, | |
| 3022 | + subpat: None, | |
| 3023 | + }); | |
| 3024 | + self.bind_pattern(&pat, it, i) | |
| 3025 | + } | |
| 3026 | + | |
| 2823 | 3027 | /// Bind a `for` pattern against a resolved iterator at index `i`. |
| 2824 | 3028 | fn bind_pattern(&mut self, p: &Pat, it: &Iter, i: &str) -> Result<(), String> { |
| 2825 | 3029 | match (p, it) { |
| @@ -2839,6 +3043,53 @@ impl Lowerer { | ||
| 2839 | 3043 | (_, Iter::Zip(..) | Iter::Enumerate(..)) => Err( |
| 2840 | 3044 | "a `zip`/`enumerate` loop needs a two-element tuple pattern".into(), |
| 2841 | 3045 | ), |
| 3046 | + // `.map`/`.filter` bind the inner element first, then transform | |
| 3047 | + // or test it. | |
| 3048 | + (_, Iter::Map { inner, param, body }) => { | |
| 3049 | + // The element is bound once under a fresh name and the | |
| 3050 | + // closure's parameter aliases it: two adaptors in one chain | |
| 3051 | + // may name their parameters the same, and each binding its own | |
| 3052 | + // `let` would collide in a single Nim scope. | |
| 3053 | + let (code, ty) = self.bind_element(inner, i)?; | |
| 3054 | + self.push_scope(); | |
| 3055 | + self.bind_alias(param, Alias::Value { code, ty }); | |
| 3056 | + let v = self.expr(body); | |
| 3057 | + self.pop_scope(); | |
| 3058 | + let v = v?; | |
| 3059 | + match p { | |
| 3060 | + Pat::Ident(id) => { | |
| 3061 | + let n = id.ident.to_string(); | |
| 3062 | + let t = v.ty.clone().ok_or("cannot infer a `map` element type")?; | |
| 3063 | + self.line(&format!("let {}: {} = {}", ident(&n), t.render(), v.code)); | |
| 3064 | + self.bind(&n, t); | |
| 3065 | + Ok(()) | |
| 3066 | + } | |
| 3067 | + Pat::Wild(_) => Ok(()), | |
| 3068 | + _ => Err("unsupported `for` pattern over `map`".into()), | |
| 3069 | + } | |
| 3070 | + } | |
| 3071 | + (_, Iter::Filter { inner, param, body }) => { | |
| 3072 | + let (code, ty) = self.bind_element(inner, i)?; | |
| 3073 | + self.push_scope(); | |
| 3074 | + self.bind_alias(param, Alias::Value { code: code.clone(), ty: ty.clone() }); | |
| 3075 | + let v = self.expr(body); | |
| 3076 | + self.pop_scope(); | |
| 3077 | + let v = v?; | |
| 3078 | + // Rust skips rejected elements; `continue` is the same thing. | |
| 3079 | + self.line(&format!("if not ({}):", v.code)); | |
| 3080 | + self.line(" continue"); | |
| 3081 | + match p { | |
| 3082 | + Pat::Ident(id) => { | |
| 3083 | + self.bind_alias( | |
| 3084 | + &id.ident.to_string(), | |
| 3085 | + Alias::Value { code, ty }, | |
| 3086 | + ); | |
| 3087 | + Ok(()) | |
| 3088 | + } | |
| 3089 | + Pat::Wild(_) => Ok(()), | |
| 3090 | + _ => Err("unsupported `for` pattern over `filter`".into()), | |
| 3091 | + } | |
| 3092 | + } | |
| 2842 | 3093 | (Pat::Wild(_), _) => Ok(()), |
| 2843 | 3094 | // `for &byte in xs` — the `&` destructures the reference, which in |
| 2844 | 3095 | // Nim is already the value. |
| @@ -2913,9 +3164,9 @@ impl Lowerer { | ||
| 2913 | 3164 | ); |
| 2914 | 3165 | Ok(()) |
| 2915 | 3166 | } |
| 2916 | - // Handled above: a zip or enumerate needs a tuple pattern, | |
| 2917 | - // and binding one name to the pair is not supported. | |
| 2918 | - Iter::Zip(..) | Iter::Enumerate(..) => unreachable!(), | |
| 3167 | + // Handled above. | |
| 3168 | + Iter::Zip(..) | Iter::Enumerate(..) | Iter::Map { .. } | |
| 3169 | + | Iter::Filter { .. } | Iter::Closure { .. } => unreachable!(), | |
| 2919 | 3170 | } |
| 2920 | 3171 | } |
| 2921 | 3172 | _ => Err("unsupported `for` pattern".into()), |
| @@ -3983,6 +4234,20 @@ impl Lowerer { | ||
| 3983 | 4234 | /// The early return is statements, not an expression, so they are emitted |
| 3984 | 4235 | /// ahead of the line being built. Every caller lowers its sub-expressions |
| 3985 | 4236 | /// before emitting its own line, which is what makes that ordering hold. |
| 4237 | + /// The element type of an expression that is itself an iterator. | |
| 4238 | + fn iter_value_type(&mut self, e: &Expr) -> Option<Nim> { | |
| 4239 | + // Only a call can produce one here, and typing it must not emit | |
| 4240 | + // anything, so the signature is consulted rather than the expression | |
| 4241 | + // lowered. | |
| 4242 | + let Expr::Call(c) = e else { return None }; | |
| 4243 | + let Expr::Path(p) = &*c.func else { return None }; | |
| 4244 | + let k = self.resolve_fn(&p.path)?; | |
| 4245 | + match &self.fns.get(&k)?.ret { | |
| 4246 | + Nim::Iter(t) => Some((**t).clone()), | |
| 4247 | + _ => None, | |
| 4248 | + } | |
| 4249 | + } | |
| 4250 | + | |
| 3986 | 4251 | /// The container, start offset, length and element type an expression |
| 3987 | 4252 | /// denotes as a slice. A window alias contributes its own offset, so |
| 3988 | 4253 | /// `dst.get_mut(..n)` followed by `.chunks_exact_mut(2)` indexes straight |
| @@ -83,6 +83,15 @@ enum Iter { | |||
| 83 | Enumerate(Box<Iter>), | 83 | Enumerate(Box<Iter>), |
| 84 | /// `.zip(other)` — stops at the shorter, as Rust's does. | 84 | /// `.zip(other)` — stops at the shorter, as Rust's does. |
| 85 | Zip(Box<Iter>, Box<Iter>), | 85 | Zip(Box<Iter>, Box<Iter>), |
| 86 | + /// `.map(|x| ..)`: the element is the closure's body, with its parameter | ||
| 87 | + /// bound to the incoming element. | ||
| 88 | + Map { inner: Box<Iter>, param: String, body: Box<Expr> }, | ||
| 89 | + /// A Nim closure iterator value, iterated directly rather than by index. | ||
| 90 | + Closure { code: String, elem: Option<Nim> }, | ||
| 91 | + /// `.filter(|x| ..)`: elements the predicate rejects are skipped. In a | ||
| 92 | + /// lazy iterator this is exact; it is only in an index loop that it | ||
| 93 | + /// changes the loop's shape. | ||
| 94 | + Filter { inner: Box<Iter>, param: String, body: Box<Expr> }, | ||
| 86 | } | 95 | } |
| 87 | 96 | ||
| 88 | impl Iter { | 97 | impl Iter { |
| @@ -98,6 +107,9 @@ impl Iter { | |||
| 98 | Iter::Chunks { k, len, .. } => format!("({} div int({}))", len, k), | 107 | Iter::Chunks { k, len, .. } => format!("({} div int({}))", len, k), |
| 99 | Iter::Windows { len, k, .. } => format!("(max(0, {} - int({}) + 1))", len, k), | 108 | Iter::Windows { len, k, .. } => format!("(max(0, {} - int({}) + 1))", len, k), |
| 100 | Iter::Enumerate(i) => i.len(), | 109 | Iter::Enumerate(i) => i.len(), |
| 110 | + Iter::Map { inner, .. } | Iter::Filter { inner, .. } => inner.len(), | ||
| 111 | + // A closure iterator has no length; it is iterated directly. | ||
| 112 | + Iter::Closure { .. } => "0".into(), | ||
| 101 | Iter::Zip(a, b) => format!("min({}, {})", a.len(), b.len()), | 113 | Iter::Zip(a, b) => format!("min({}, {})", a.len(), b.len()), |
| 102 | } | 114 | } |
| 103 | } | 115 | } |
| @@ -2171,6 +2183,31 @@ impl Lowerer { | |||
| 2171 | } else { | 2183 | } else { |
| 2172 | self.target.replace(("result".to_string(), Some(ret.clone()))) | 2184 | self.target.replace(("result".to_string(), Some(ret.clone()))) |
| 2173 | }; | 2185 | }; |
| 2186 | + // `-> impl Iterator<Item = T>` becomes a Nim closure iterator, which | ||
| 2187 | + // is lazy and first-class as Rust's is. The body's trailing expression | ||
| 2188 | + // is the chain it yields from. | ||
| 2189 | + if let Nim::Iter(elem) = &ret { | ||
| 2190 | + let elem = (**elem).clone(); | ||
| 2191 | + // The *declared* parameter types, not the scope's, which record | ||
| 2192 | + // the owned form. | ||
| 2193 | + let views: Vec<(String, Nim)> = typed | ||
| 2194 | + .iter() | ||
| 2195 | + .zip(ptys.iter()) | ||
| 2196 | + .filter(|(_, t)| matches!(t, Nim::OpenArray(_))) | ||
| 2197 | + .filter_map(|(p, t)| match &*p.pat { | ||
| 2198 | + Pat::Ident(i) => Some((i.ident.to_string(), t.clone())), | ||
| 2199 | + _ => None, | ||
| 2200 | + }) | ||
| 2201 | + .collect(); | ||
| 2202 | + let r = self.iter_body(body, &elem, &views); | ||
| 2203 | + self.indent -= 1; | ||
| 2204 | + self.ret = outer_ret; | ||
| 2205 | + self.fn_generics = outer_fg; | ||
| 2206 | + self.target = outer_target; | ||
| 2207 | + self.pop_scope(); | ||
| 2208 | + self.blank(); | ||
| 2209 | + return r; | ||
| 2210 | + } | ||
| 2174 | let before = self.out.len(); | 2211 | let before = self.out.len(); |
| 2175 | let tail = self.block_body_at(body, Some(&ret))?; | 2212 | let tail = self.block_body_at(body, Some(&ret))?; |
| 2176 | self.target = outer_target; | 2213 | self.target = outer_target; |
| @@ -2687,6 +2724,34 @@ impl Lowerer { | |||
| 2687 | } | 2724 | } |
| 2688 | let it = self.resolve_iter(&f.expr)?; | 2725 | let it = self.resolve_iter(&f.expr)?; |
| 2689 | 2726 | ||
| 2727 | + // A closure iterator is iterated directly; there is no index. | ||
| 2728 | + if let Iter::Closure { code, elem } = &it { | ||
| 2729 | + let name = match &*f.pat { | ||
| 2730 | + Pat::Ident(i) => i.ident.to_string(), | ||
| 2731 | + Pat::Wild(_) => "_".into(), | ||
| 2732 | + _ => return Err("unsupported `for` pattern over an iterator".into()), | ||
| 2733 | + }; | ||
| 2734 | + self.line(&format!("for {} in {}():", ident(&name), code)); | ||
| 2735 | + self.indent += 1; | ||
| 2736 | + self.push_scope(); | ||
| 2737 | + if let Some(t) = elem { | ||
| 2738 | + self.bind(&name, t.clone()); | ||
| 2739 | + } | ||
| 2740 | + let before = self.out.len(); | ||
| 2741 | + let saved = self.target.take(); | ||
| 2742 | + if let Some(v) = self.block_body(&f.body)? { | ||
| 2743 | + let code = v.code.clone(); | ||
| 2744 | + self.line(&format!("discard {code}")); | ||
| 2745 | + } | ||
| 2746 | + self.target = saved; | ||
| 2747 | + if self.out.len() == before { | ||
| 2748 | + self.line("discard"); | ||
| 2749 | + } | ||
| 2750 | + self.pop_scope(); | ||
| 2751 | + self.indent -= 1; | ||
| 2752 | + return Ok(()); | ||
| 2753 | + } | ||
| 2754 | + | ||
| 2690 | // One index loop drives the whole chain. Rust's adaptors are lazy and | 2755 | // One index loop drives the whole chain. Rust's adaptors are lazy and |
| 2691 | // compose; resolving them to an index and binding each name to an | 2756 | // compose; resolving them to an index and binding each name to an |
| 2692 | // lvalue reproduces that without materialising anything. | 2757 | // lvalue reproduces that without materialising anything. |
| @@ -2753,6 +2818,39 @@ impl Lowerer { | |||
| 2753 | } | 2818 | } |
| 2754 | Ok(it) | 2819 | Ok(it) |
| 2755 | } | 2820 | } |
| 2821 | + "map" | "filter" if m.args.len() == 1 => { | ||
| 2822 | + let Expr::Closure(c) = &m.args[0] else { | ||
| 2823 | + return Err(format!( | ||
| 2824 | + "`.{name}()` needs a closure written at the call \ | ||
| 2825 | + site; a function value would need its element \ | ||
| 2826 | + type, which is not tracked here" | ||
| 2827 | + )); | ||
| 2828 | + }; | ||
| 2829 | + if c.inputs.len() != 1 { | ||
| 2830 | + return Err(format!("`.{name}()` takes a one-argument closure")); | ||
| 2831 | + } | ||
| 2832 | + let param = match &c.inputs[0] { | ||
| 2833 | + Pat::Ident(i) => i.ident.to_string(), | ||
| 2834 | + Pat::Wild(_) => "unused0".into(), | ||
| 2835 | + Pat::Reference(r) => match &*r.pat { | ||
| 2836 | + Pat::Ident(i) => i.ident.to_string(), | ||
| 2837 | + _ => return Err("unsupported closure parameter".into()), | ||
| 2838 | + }, | ||
| 2839 | + _ => return Err("unsupported closure parameter".into()), | ||
| 2840 | + }; | ||
| 2841 | + let body = match &*c.body { | ||
| 2842 | + Expr::Block(b) => single_expr(&b.block) | ||
| 2843 | + .ok_or("a closure body with statements is not implemented yet")? | ||
| 2844 | + .clone(), | ||
| 2845 | + other => other.clone(), | ||
| 2846 | + }; | ||
| 2847 | + let inner = Box::new(self.resolve_iter(&m.receiver)?); | ||
| 2848 | + Ok(if name == "map" { | ||
| 2849 | + Iter::Map { inner, param, body: Box::new(body) } | ||
| 2850 | + } else { | ||
| 2851 | + Iter::Filter { inner, param, body: Box::new(body) } | ||
| 2852 | + }) | ||
| 2853 | + } | ||
| 2756 | "enumerate" if m.args.is_empty() => { | 2854 | "enumerate" if m.args.is_empty() => { |
| 2757 | Ok(Iter::Enumerate(Box::new(self.resolve_iter(&m.receiver)?))) | 2855 | Ok(Iter::Enumerate(Box::new(self.resolve_iter(&m.receiver)?))) |
| 2758 | } | 2856 | } |
| @@ -2790,6 +2888,14 @@ impl Lowerer { | |||
| 2790 | if let Some(Alias::Iterator(it)) = self.lookup_alias(&n) { | 2888 | if let Some(Alias::Iterator(it)) = self.lookup_alias(&n) { |
| 2791 | return Ok((*it).clone()); | 2889 | return Ok((*it).clone()); |
| 2792 | } | 2890 | } |
| 2891 | + // A binding whose type *is* an iterator -- a parameter of | ||
| 2892 | + // `impl Iterator` type, say -- is iterated directly. | ||
| 2893 | + if let Some(Nim::Iter(elem)) = self.lookup(&n) { | ||
| 2894 | + return Ok(Iter::Closure { | ||
| 2895 | + code: ident(&n), | ||
| 2896 | + elem: Some((*elem).clone()), | ||
| 2897 | + }); | ||
| 2898 | + } | ||
| 2793 | if let Some(Alias::Window { code, off, len, elem }) = self.lookup_alias(&n) { | 2899 | if let Some(Alias::Window { code, off, len, elem }) = self.lookup_alias(&n) { |
| 2794 | return Ok(Iter::Elems { code, off, len, elem, mutable: false }); | 2900 | return Ok(Iter::Elems { code, off, len, elem, mutable: false }); |
| 2795 | } | 2901 | } |
| @@ -2802,6 +2908,17 @@ impl Lowerer { | |||
| 2802 | mutable: false, | 2908 | mutable: false, |
| 2803 | }) | 2909 | }) |
| 2804 | } | 2910 | } |
| 2911 | + other if matches!(self.iter_value_type(other), Some(_)) => { | ||
| 2912 | + // A value that *is* an iterator -- typically a call returning | ||
| 2913 | + // `impl Iterator`. It must be bound before being iterated: | ||
| 2914 | + // `for x in f()()` would rebuild it on every step and never | ||
| 2915 | + // terminate. Probed against Nim 2.2.4. | ||
| 2916 | + let elem = self.iter_value_type(other).unwrap(); | ||
| 2917 | + let v = self.expr(other)?; | ||
| 2918 | + let tmp = self.fresh("It"); | ||
| 2919 | + self.line(&format!("let {} = {}", tmp, v.code)); | ||
| 2920 | + Ok(Iter::Closure { code: tmp, elem: Some(elem) }) | ||
| 2921 | + } | ||
| 2805 | other => { | 2922 | other => { |
| 2806 | // A `for` binding that is itself a window iterates that window, | 2923 | // A `for` binding that is itself a window iterates that window, |
| 2807 | // not the whole container it points into. | 2924 | // not the whole container it points into. |
| @@ -2820,6 +2937,93 @@ impl Lowerer { | |||
| 2820 | } | 2937 | } |
| 2821 | } | 2938 | } |
| 2822 | 2939 | ||
| 2940 | + /// The body of a function returning `impl Iterator`: everything but the | ||
| 2941 | + /// trailing expression runs once, then the chain becomes a closure | ||
| 2942 | + /// iterator that yields. | ||
| 2943 | + fn iter_body( | ||
| 2944 | + &mut self, | ||
| 2945 | + body: &syn::Block, | ||
| 2946 | + elem: &Nim, | ||
| 2947 | + views: &[(String, Nim)], | ||
| 2948 | + ) -> Result<(), String> { | ||
| 2949 | + let n = body.stmts.len(); | ||
| 2950 | + let Some(Stmt::Expr(last, None)) = body.stmts.last() else { | ||
| 2951 | + return Err("a function returning `impl Iterator` must end in the \ | ||
| 2952 | + iterator expression" | ||
| 2953 | + .into()); | ||
| 2954 | + }; | ||
| 2955 | + for st in &body.stmts[..n - 1] { | ||
| 2956 | + self.stmt(st)?; | ||
| 2957 | + } | ||
| 2958 | + // Nim will not let a closure capture an `openArray`: it is a view, | ||
| 2959 | + // and capturing it could outlive what it points at. That is the same | ||
| 2960 | + // property Rust states with `+ '_`. Each such parameter is copied once | ||
| 2961 | + // per call into a `seq` the closure can own. Rust guarantees the | ||
| 2962 | + // borrow outlives the iterator, so nothing can mutate the original | ||
| 2963 | + // while it is being read, which is what makes the copy unobservable -- | ||
| 2964 | + // it is a cost, not a semantic change. | ||
| 2965 | + for (n, t) in views { | ||
| 2966 | + let copy = self.fresh("Cap"); | ||
| 2967 | + let elem_t = match t { | ||
| 2968 | + Nim::OpenArray(e) => (**e).clone(), | ||
| 2969 | + other => other.clone(), | ||
| 2970 | + }; | ||
| 2971 | + self.line(&format!("let {}: seq[{}] = @{}", copy, elem_t.render(), ident(n))); | ||
| 2972 | + self.bind_alias( | ||
| 2973 | + n, | ||
| 2974 | + Alias::Value { code: copy, ty: Some(Nim::Seq(Box::new(elem_t))) }, | ||
| 2975 | + ); | ||
| 2976 | + } | ||
| 2977 | + | ||
| 2978 | + let it = self.resolve_iter(last)?; | ||
| 2979 | + self.line(&format!( | ||
| 2980 | + "result = iterator (): {} {{.closure.}} =", | ||
| 2981 | + elem.render() | ||
| 2982 | + )); | ||
| 2983 | + self.indent += 1; | ||
| 2984 | + let i = self.fresh("Idx"); | ||
| 2985 | + self.line(&format!("for {} in 0 ..< int({}):", i, it.len())); | ||
| 2986 | + self.indent += 1; | ||
| 2987 | + self.push_scope(); | ||
| 2988 | + let name = self.fresh("Elem"); | ||
| 2989 | + self.bind_pattern_named(&name, &it, &i)?; | ||
| 2990 | + // The binding may be an alias rather than a Nim variable, so the | ||
| 2991 | + // yielded expression is looked up rather than assumed to be the name. | ||
| 2992 | + let v = match self.lookup_alias(&name) { | ||
| 2993 | + Some(Alias::Value { code, .. }) => code, | ||
| 2994 | + _ => ident(&name), | ||
| 2995 | + }; | ||
| 2996 | + self.line(&format!("yield {}", v)); | ||
| 2997 | + self.pop_scope(); | ||
| 2998 | + self.indent -= 2; | ||
| 2999 | + Ok(()) | ||
| 3000 | + } | ||
| 3001 | + | ||
| 3002 | + /// Bind one element of an iterator under a fresh name, and return how to | ||
| 3003 | + /// refer to it and what type it has. | ||
| 3004 | + fn bind_element(&mut self, it: &Iter, i: &str) -> Result<(String, Option<Nim>), String> { | ||
| 3005 | + let n = self.fresh("El"); | ||
| 3006 | + self.bind_pattern_named(&n, it, i)?; | ||
| 3007 | + Ok(match self.lookup_alias(&n) { | ||
| 3008 | + Some(Alias::Value { code, ty }) => (code, ty), | ||
| 3009 | + _ => (ident(&n), self.lookup(&n)), | ||
| 3010 | + }) | ||
| 3011 | + } | ||
| 3012 | + | ||
| 3013 | + /// Bind one name against a resolved iterator, for a closure parameter. | ||
| 3014 | + fn bind_pattern_named(&mut self, name: &str, it: &Iter, i: &str) -> Result<(), String> { | ||
| 3015 | + let id: syn::Ident = | ||
| 3016 | + syn::parse_str(name).map_err(|e: syn::Error| e.to_string())?; | ||
| 3017 | + let pat = Pat::Ident(syn::PatIdent { | ||
| 3018 | + attrs: Vec::new(), | ||
| 3019 | + by_ref: None, | ||
| 3020 | + mutability: None, | ||
| 3021 | + ident: id, | ||
| 3022 | + subpat: None, | ||
| 3023 | + }); | ||
| 3024 | + self.bind_pattern(&pat, it, i) | ||
| 3025 | + } | ||
| 3026 | + | ||
| 2823 | /// Bind a `for` pattern against a resolved iterator at index `i`. | 3027 | /// Bind a `for` pattern against a resolved iterator at index `i`. |
| 2824 | fn bind_pattern(&mut self, p: &Pat, it: &Iter, i: &str) -> Result<(), String> { | 3028 | fn bind_pattern(&mut self, p: &Pat, it: &Iter, i: &str) -> Result<(), String> { |
| 2825 | match (p, it) { | 3029 | match (p, it) { |
| @@ -2839,6 +3043,53 @@ impl Lowerer { | |||
| 2839 | (_, Iter::Zip(..) | Iter::Enumerate(..)) => Err( | 3043 | (_, Iter::Zip(..) | Iter::Enumerate(..)) => Err( |
| 2840 | "a `zip`/`enumerate` loop needs a two-element tuple pattern".into(), | 3044 | "a `zip`/`enumerate` loop needs a two-element tuple pattern".into(), |
| 2841 | ), | 3045 | ), |
| 3046 | + // `.map`/`.filter` bind the inner element first, then transform | ||
| 3047 | + // or test it. | ||
| 3048 | + (_, Iter::Map { inner, param, body }) => { | ||
| 3049 | + // The element is bound once under a fresh name and the | ||
| 3050 | + // closure's parameter aliases it: two adaptors in one chain | ||
| 3051 | + // may name their parameters the same, and each binding its own | ||
| 3052 | + // `let` would collide in a single Nim scope. | ||
| 3053 | + let (code, ty) = self.bind_element(inner, i)?; | ||
| 3054 | + self.push_scope(); | ||
| 3055 | + self.bind_alias(param, Alias::Value { code, ty }); | ||
| 3056 | + let v = self.expr(body); | ||
| 3057 | + self.pop_scope(); | ||
| 3058 | + let v = v?; | ||
| 3059 | + match p { | ||
| 3060 | + Pat::Ident(id) => { | ||
| 3061 | + let n = id.ident.to_string(); | ||
| 3062 | + let t = v.ty.clone().ok_or("cannot infer a `map` element type")?; | ||
| 3063 | + self.line(&format!("let {}: {} = {}", ident(&n), t.render(), v.code)); | ||
| 3064 | + self.bind(&n, t); | ||
| 3065 | + Ok(()) | ||
| 3066 | + } | ||
| 3067 | + Pat::Wild(_) => Ok(()), | ||
| 3068 | + _ => Err("unsupported `for` pattern over `map`".into()), | ||
| 3069 | + } | ||
| 3070 | + } | ||
| 3071 | + (_, Iter::Filter { inner, param, body }) => { | ||
| 3072 | + let (code, ty) = self.bind_element(inner, i)?; | ||
| 3073 | + self.push_scope(); | ||
| 3074 | + self.bind_alias(param, Alias::Value { code: code.clone(), ty: ty.clone() }); | ||
| 3075 | + let v = self.expr(body); | ||
| 3076 | + self.pop_scope(); | ||
| 3077 | + let v = v?; | ||
| 3078 | + // Rust skips rejected elements; `continue` is the same thing. | ||
| 3079 | + self.line(&format!("if not ({}):", v.code)); | ||
| 3080 | + self.line(" continue"); | ||
| 3081 | + match p { | ||
| 3082 | + Pat::Ident(id) => { | ||
| 3083 | + self.bind_alias( | ||
| 3084 | + &id.ident.to_string(), | ||
| 3085 | + Alias::Value { code, ty }, | ||
| 3086 | + ); | ||
| 3087 | + Ok(()) | ||
| 3088 | + } | ||
| 3089 | + Pat::Wild(_) => Ok(()), | ||
| 3090 | + _ => Err("unsupported `for` pattern over `filter`".into()), | ||
| 3091 | + } | ||
| 3092 | + } | ||
| 2842 | (Pat::Wild(_), _) => Ok(()), | 3093 | (Pat::Wild(_), _) => Ok(()), |
| 2843 | // `for &byte in xs` — the `&` destructures the reference, which in | 3094 | // `for &byte in xs` — the `&` destructures the reference, which in |
| 2844 | // Nim is already the value. | 3095 | // Nim is already the value. |
| @@ -2913,9 +3164,9 @@ impl Lowerer { | |||
| 2913 | ); | 3164 | ); |
| 2914 | Ok(()) | 3165 | Ok(()) |
| 2915 | } | 3166 | } |
| 2916 | - // Handled above: a zip or enumerate needs a tuple pattern, | 3167 | + // Handled above. |
| 2917 | - // and binding one name to the pair is not supported. | 3168 | + Iter::Zip(..) | Iter::Enumerate(..) | Iter::Map { .. } |
| 2918 | - Iter::Zip(..) | Iter::Enumerate(..) => unreachable!(), | 3169 | + | Iter::Filter { .. } | Iter::Closure { .. } => unreachable!(), |
| 2919 | } | 3170 | } |
| 2920 | } | 3171 | } |
| 2921 | _ => Err("unsupported `for` pattern".into()), | 3172 | _ => Err("unsupported `for` pattern".into()), |
| @@ -3983,6 +4234,20 @@ impl Lowerer { | |||
| 3983 | /// The early return is statements, not an expression, so they are emitted | 4234 | /// The early return is statements, not an expression, so they are emitted |
| 3984 | /// ahead of the line being built. Every caller lowers its sub-expressions | 4235 | /// ahead of the line being built. Every caller lowers its sub-expressions |
| 3985 | /// before emitting its own line, which is what makes that ordering hold. | 4236 | /// before emitting its own line, which is what makes that ordering hold. |
| 4237 | + /// The element type of an expression that is itself an iterator. | ||
| 4238 | + fn iter_value_type(&mut self, e: &Expr) -> Option<Nim> { | ||
| 4239 | + // Only a call can produce one here, and typing it must not emit | ||
| 4240 | + // anything, so the signature is consulted rather than the expression | ||
| 4241 | + // lowered. | ||
| 4242 | + let Expr::Call(c) = e else { return None }; | ||
| 4243 | + let Expr::Path(p) = &*c.func else { return None }; | ||
| 4244 | + let k = self.resolve_fn(&p.path)?; | ||
| 4245 | + match &self.fns.get(&k)?.ret { | ||
| 4246 | + Nim::Iter(t) => Some((**t).clone()), | ||
| 4247 | + _ => None, | ||
| 4248 | + } | ||
| 4249 | + } | ||
| 4250 | + | ||
| 3986 | /// The container, start offset, length and element type an expression | 4251 | /// The container, start offset, length and element type an expression |
| 3987 | /// denotes as a slice. A window alias contributes its own offset, so | 4252 | /// denotes as a slice. A window alias contributes its own offset, so |
| 3988 | /// `dst.get_mut(..n)` followed by `.chunks_exact_mut(2)` indexes straight | 4253 | /// `dst.get_mut(..n)` followed by `.chunks_exact_mut(2)` indexes straight |
modified
src/ty.rs +32 -0 | @@ -18,6 +18,9 @@ pub enum Nim { | ||
| 18 | 18 | Var(Box<Nim>), |
| 19 | 19 | /// `*mut T`. Nim's `ptr` is the same thing: an unmanaged address. |
| 20 | 20 | Ptr(Box<Nim>), |
| 21 | + /// `-> impl Iterator<Item = T>`. Nim's closure iterator is the same | |
| 22 | + /// thing: lazy, first-class, and resumable. | |
| 23 | + Iter(Box<Nim>), | |
| 21 | 24 | /// `*const T`. The const matters at the C level even though it does not at |
| 22 | 25 | /// the Rust one: Nim emits a real prototype where Rust emits none, and a |
| 23 | 26 | /// `char *` declaration against C's `const char *` is a compile error. |
| @@ -50,6 +53,7 @@ impl Nim { | ||
| 50 | 53 | Nim::Prim(p) if p == "void" => "pointer".into(), |
| 51 | 54 | inner => format!("ptr {}", inner.render()), |
| 52 | 55 | }, |
| 56 | + Nim::Iter(t) => format!("iterator (): {} {{.closure.}}", t.render()), | |
| 53 | 57 | Nim::ConstPtr(t) => match &**t { |
| 54 | 58 | Nim::Prim(p) if p == "void" => "pointer".into(), |
| 55 | 59 | inner => const_ptr_alias(inner), |
| @@ -336,6 +340,20 @@ pub fn map(t: &Type) -> Result<Nim, String> { | ||
| 336 | 340 | if let TypeParamBound::Trait(tb) = b { |
| 337 | 341 | if let Some(seg) = tb.path.segments.last() { |
| 338 | 342 | let n = seg.ident.to_string(); |
| 343 | + // `impl Iterator<Item = T>` in return position: Nim's | |
| 344 | + // closure iterator is lazy and first-class, so this is | |
| 345 | + // a real equivalent rather than a materialised `seq`. | |
| 346 | + if n == "Iterator" || n == "IntoIterator" { | |
| 347 | + if let PathArguments::AngleBracketed(a) = &seg.arguments { | |
| 348 | + for g in &a.args { | |
| 349 | + if let GenericArgument::AssocType(at) = g { | |
| 350 | + if at.ident == "Item" { | |
| 351 | + return Ok(Nim::Iter(Box::new(map(&at.ty)?))); | |
| 352 | + } | |
| 353 | + } | |
| 354 | + } | |
| 355 | + } | |
| 356 | + } | |
| 339 | 357 | if n == "Fn" || n == "FnMut" || n == "FnOnce" { |
| 340 | 358 | if let PathArguments::Parenthesized(a) = &seg.arguments { |
| 341 | 359 | let args: Vec<Nim> = a |
| @@ -362,6 +380,20 @@ pub fn map(t: &Type) -> Result<Nim, String> { | ||
| 362 | 380 | for b in &i.bounds { |
| 363 | 381 | if let TypeParamBound::Trait(tb) = b { |
| 364 | 382 | if let Some(seg) = tb.path.segments.last() { |
| 383 | + // `impl Iterator<Item = T>`: Nim's closure iterator is | |
| 384 | + // lazy and first-class as Rust's is, so this is a real | |
| 385 | + // equivalent rather than a materialised `seq`. | |
| 386 | + if seg.ident == "Iterator" || seg.ident == "IntoIterator" { | |
| 387 | + if let PathArguments::AngleBracketed(a) = &seg.arguments { | |
| 388 | + for g in &a.args { | |
| 389 | + if let GenericArgument::AssocType(at) = g { | |
| 390 | + if at.ident == "Item" { | |
| 391 | + return Ok(Nim::Iter(Box::new(map(&at.ty)?))); | |
| 392 | + } | |
| 393 | + } | |
| 394 | + } | |
| 395 | + } | |
| 396 | + } | |
| 365 | 397 | // `impl Fn(A) -> B` is a callable; Nim has a proc type |
| 366 | 398 | // for exactly this. |
| 367 | 399 | if seg.ident == "Fn" || seg.ident == "FnMut" || seg.ident == "FnOnce" { |
| @@ -18,6 +18,9 @@ pub enum Nim { | |||
| 18 | Var(Box<Nim>), | 18 | Var(Box<Nim>), |
| 19 | /// `*mut T`. Nim's `ptr` is the same thing: an unmanaged address. | 19 | /// `*mut T`. Nim's `ptr` is the same thing: an unmanaged address. |
| 20 | Ptr(Box<Nim>), | 20 | Ptr(Box<Nim>), |
| 21 | + /// `-> impl Iterator<Item = T>`. Nim's closure iterator is the same | ||
| 22 | + /// thing: lazy, first-class, and resumable. | ||
| 23 | + Iter(Box<Nim>), | ||
| 21 | /// `*const T`. The const matters at the C level even though it does not at | 24 | /// `*const T`. The const matters at the C level even though it does not at |
| 22 | /// the Rust one: Nim emits a real prototype where Rust emits none, and a | 25 | /// the Rust one: Nim emits a real prototype where Rust emits none, and a |
| 23 | /// `char *` declaration against C's `const char *` is a compile error. | 26 | /// `char *` declaration against C's `const char *` is a compile error. |
| @@ -50,6 +53,7 @@ impl Nim { | |||
| 50 | Nim::Prim(p) if p == "void" => "pointer".into(), | 53 | Nim::Prim(p) if p == "void" => "pointer".into(), |
| 51 | inner => format!("ptr {}", inner.render()), | 54 | inner => format!("ptr {}", inner.render()), |
| 52 | }, | 55 | }, |
| 56 | + Nim::Iter(t) => format!("iterator (): {} {{.closure.}}", t.render()), | ||
| 53 | Nim::ConstPtr(t) => match &**t { | 57 | Nim::ConstPtr(t) => match &**t { |
| 54 | Nim::Prim(p) if p == "void" => "pointer".into(), | 58 | Nim::Prim(p) if p == "void" => "pointer".into(), |
| 55 | inner => const_ptr_alias(inner), | 59 | inner => const_ptr_alias(inner), |
| @@ -336,6 +340,20 @@ pub fn map(t: &Type) -> Result<Nim, String> { | |||
| 336 | if let TypeParamBound::Trait(tb) = b { | 340 | if let TypeParamBound::Trait(tb) = b { |
| 337 | if let Some(seg) = tb.path.segments.last() { | 341 | if let Some(seg) = tb.path.segments.last() { |
| 338 | let n = seg.ident.to_string(); | 342 | let n = seg.ident.to_string(); |
| 343 | + // `impl Iterator<Item = T>` in return position: Nim's | ||
| 344 | + // closure iterator is lazy and first-class, so this is | ||
| 345 | + // a real equivalent rather than a materialised `seq`. | ||
| 346 | + if n == "Iterator" || n == "IntoIterator" { | ||
| 347 | + if let PathArguments::AngleBracketed(a) = &seg.arguments { | ||
| 348 | + for g in &a.args { | ||
| 349 | + if let GenericArgument::AssocType(at) = g { | ||
| 350 | + if at.ident == "Item" { | ||
| 351 | + return Ok(Nim::Iter(Box::new(map(&at.ty)?))); | ||
| 352 | + } | ||
| 353 | + } | ||
| 354 | + } | ||
| 355 | + } | ||
| 356 | + } | ||
| 339 | if n == "Fn" || n == "FnMut" || n == "FnOnce" { | 357 | if n == "Fn" || n == "FnMut" || n == "FnOnce" { |
| 340 | if let PathArguments::Parenthesized(a) = &seg.arguments { | 358 | if let PathArguments::Parenthesized(a) = &seg.arguments { |
| 341 | let args: Vec<Nim> = a | 359 | let args: Vec<Nim> = a |
| @@ -362,6 +380,20 @@ pub fn map(t: &Type) -> Result<Nim, String> { | |||
| 362 | for b in &i.bounds { | 380 | for b in &i.bounds { |
| 363 | if let TypeParamBound::Trait(tb) = b { | 381 | if let TypeParamBound::Trait(tb) = b { |
| 364 | if let Some(seg) = tb.path.segments.last() { | 382 | if let Some(seg) = tb.path.segments.last() { |
| 383 | + // `impl Iterator<Item = T>`: Nim's closure iterator is | ||
| 384 | + // lazy and first-class as Rust's is, so this is a real | ||
| 385 | + // equivalent rather than a materialised `seq`. | ||
| 386 | + if seg.ident == "Iterator" || seg.ident == "IntoIterator" { | ||
| 387 | + if let PathArguments::AngleBracketed(a) = &seg.arguments { | ||
| 388 | + for g in &a.args { | ||
| 389 | + if let GenericArgument::AssocType(at) = g { | ||
| 390 | + if at.ident == "Item" { | ||
| 391 | + return Ok(Nim::Iter(Box::new(map(&at.ty)?))); | ||
| 392 | + } | ||
| 393 | + } | ||
| 394 | + } | ||
| 395 | + } | ||
| 396 | + } | ||
| 365 | // `impl Fn(A) -> B` is a callable; Nim has a proc type | 397 | // `impl Fn(A) -> B` is a callable; Nim has a proc type |
| 366 | // for exactly this. | 398 | // for exactly this. |
| 367 | if seg.ident == "Fn" || seg.ident == "FnMut" || seg.ident == "FnOnce" { | 399 | if seg.ident == "Fn" || seg.ident == "FnMut" || seg.ident == "FnOnce" { |
added
tests/cases/041-return-impl-iterator.rs +56 -0 | new file mode 100644 | ||
| @@ -0,0 +1,56 @@ | ||
| 1 | +// `-> impl Iterator<Item = T>` becomes a Nim closure iterator, which is lazy | |
| 2 | +// and first-class as Rust's is -- not a materialised sequence, so an infinite | |
| 3 | +// iterator stays infinite rather than hanging. | |
| 4 | +// | |
| 5 | +// `.map` and `.filter` are exact here: in a lazy iterator they are a transform | |
| 6 | +// and a skip, with no elements computed that Rust would not compute. | |
| 7 | + | |
| 8 | +fn doubled(xs: &[i32]) -> impl Iterator<Item = i32> + '_ { | |
| 9 | + xs.iter().map(|x| x * 2) | |
| 10 | +} | |
| 11 | + | |
| 12 | +fn evens(xs: &[i32]) -> impl Iterator<Item = i32> + '_ { | |
| 13 | + xs.iter().filter(|x| *x % 2 == 0).map(|x| x * 10) | |
| 14 | +} | |
| 15 | + | |
| 16 | +fn counted(n: i32) -> impl Iterator<Item = i32> { | |
| 17 | + (0..n).map(|i| i * i) | |
| 18 | +} | |
| 19 | + | |
| 20 | +fn sum_of(it: impl Iterator<Item = i32>) -> i32 { | |
| 21 | + let mut t: i32 = 0; | |
| 22 | + for v in it { | |
| 23 | + t += v; | |
| 24 | + } | |
| 25 | + t | |
| 26 | +} | |
| 27 | + | |
| 28 | +fn main() { | |
| 29 | + let v: Vec<i32> = vec![1, 2, 3, 4, 5, 6]; | |
| 30 | + | |
| 31 | + for x in doubled(&v) { | |
| 32 | + print!("{} ", x); | |
| 33 | + } | |
| 34 | + println!(""); | |
| 35 | + | |
| 36 | + for x in evens(&v) { | |
| 37 | + print!("{} ", x); | |
| 38 | + } | |
| 39 | + println!(""); | |
| 40 | + | |
| 41 | + for x in counted(5) { | |
| 42 | + print!("{} ", x); | |
| 43 | + } | |
| 44 | + println!(""); | |
| 45 | + | |
| 46 | + // Laziness: take only the first few of a long sequence. | |
| 47 | + let mut n = 0; | |
| 48 | + for x in doubled(&v) { | |
| 49 | + print!("{} ", x); | |
| 50 | + n += 1; | |
| 51 | + if n == 3 { | |
| 52 | + break; | |
| 53 | + } | |
| 54 | + } | |
| 55 | + println!(""); | |
| 56 | +} | |
| new file mode 100644 | |||
| @@ -0,0 +1,56 @@ | |||
| 1 | +// `-> impl Iterator<Item = T>` becomes a Nim closure iterator, which is lazy | ||
| 2 | +// and first-class as Rust's is -- not a materialised sequence, so an infinite | ||
| 3 | +// iterator stays infinite rather than hanging. | ||
| 4 | +// | ||
| 5 | +// `.map` and `.filter` are exact here: in a lazy iterator they are a transform | ||
| 6 | +// and a skip, with no elements computed that Rust would not compute. | ||
| 7 | + | ||
| 8 | +fn doubled(xs: &[i32]) -> impl Iterator<Item = i32> + '_ { | ||
| 9 | + xs.iter().map(|x| x * 2) | ||
| 10 | +} | ||
| 11 | + | ||
| 12 | +fn evens(xs: &[i32]) -> impl Iterator<Item = i32> + '_ { | ||
| 13 | + xs.iter().filter(|x| *x % 2 == 0).map(|x| x * 10) | ||
| 14 | +} | ||
| 15 | + | ||
| 16 | +fn counted(n: i32) -> impl Iterator<Item = i32> { | ||
| 17 | + (0..n).map(|i| i * i) | ||
| 18 | +} | ||
| 19 | + | ||
| 20 | +fn sum_of(it: impl Iterator<Item = i32>) -> i32 { | ||
| 21 | + let mut t: i32 = 0; | ||
| 22 | + for v in it { | ||
| 23 | + t += v; | ||
| 24 | + } | ||
| 25 | + t | ||
| 26 | +} | ||
| 27 | + | ||
| 28 | +fn main() { | ||
| 29 | + let v: Vec<i32> = vec![1, 2, 3, 4, 5, 6]; | ||
| 30 | + | ||
| 31 | + for x in doubled(&v) { | ||
| 32 | + print!("{} ", x); | ||
| 33 | + } | ||
| 34 | + println!(""); | ||
| 35 | + | ||
| 36 | + for x in evens(&v) { | ||
| 37 | + print!("{} ", x); | ||
| 38 | + } | ||
| 39 | + println!(""); | ||
| 40 | + | ||
| 41 | + for x in counted(5) { | ||
| 42 | + print!("{} ", x); | ||
| 43 | + } | ||
| 44 | + println!(""); | ||
| 45 | + | ||
| 46 | + // Laziness: take only the first few of a long sequence. | ||
| 47 | + let mut n = 0; | ||
| 48 | + for x in doubled(&v) { | ||
| 49 | + print!("{} ", x); | ||
| 50 | + n += 1; | ||
| 51 | + if n == 3 { | ||
| 52 | + break; | ||
| 53 | + } | ||
| 54 | + } | ||
| 55 | + println!(""); | ||
| 56 | +} | ||