nandi/rustnimpublic Fork 0
d459281
Commits
Clone
git clone https://git.rickub.com/nandi/rustnim.git
git clone ssh://git@rickub.com/nandi/rustnim.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Expand multi-rule and recursive macro_rules

Rules are tried top to bottom and the first whose matcher matches wins, which
is Rust's rule and is also what makes a recursive macro terminate: a base
case sits above the recursive rule. Recursion then falls out for free, since
an expansion containing another macro call is lowered like any expression and
so expands again. A 64-level limit turns a macro that does not terminate into
an error rather than a hang.

That takes the expander to 1,369 of the 1,833 macro_rules! in the sample,
74%. What is left is token-tree munching (420) and nested repetition (44).

One bug found by the test and worth recording, because it was silent rather
than loud: a fragment must not span a top-level `,` or `;`. Without that a
greedy `$x:expr` swallows the whole invocation, so `describe!(3, 4)` matched
the *one-argument* rule and took the wrong branch with no error anywhere. A
comma inside brackets lives in a Group token and is not at that level at all,
which is what makes the rule safe.

The survey is unchanged at 18 of 400 accepted and 12 Nim-compilable, as it
was for repetition. Eight features running with that result. The remaining
blockers name the reason: unsupported types (86) and calls into dependencies
(49), which is the unbounded tail rather than the finite feature pool.

45 differential cases, all green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nandithebull committed 2026-09-18T23:47:31-07:00 Browse files
d459281 parent: 3d4a7d2
modified DESIGN.md +28 -16
@@ -630,25 +630,29 @@ which is exactly the guessing the project refuses. Expanding at the call site
630630 yields ordinary Rust in a context where the types are known, so it lowers like
631631 anything else. Same applicability, faithful output.
632632
633-`src/mrules.rs` implements single-rule macros, **with `$(..)` repetition**.
634-Recounting the 1,833 definitions by the expander's actual boundary — one rule,
635-any repetition, no `:tt`:
633+`src/mrules.rs` handles any number of rules, with `$(..)` repetition.
634+Measured against the 1,833 definitions in the sample:
636635
637-| | count |
638-|---|---|
639-| single rule, no repetition | 661 |
640-| single rule, with repetition | 271 |
641-| **expandable** | **932 (51%)** |
642-| multiple rules | 481 |
643-| `:tt` | 420 |
636+| | count | |
637+|---|---|---|
638+| **expandable** | **1,369** | **74%** |
639+| nested `$(..)` inside `$(..)` | 44 | 2% |
640+| `:tt` token-tree munching | 420 | 22% |
641+
642+Rules are tried top to bottom and the first whose matcher matches wins, which
643+is Rust's rule and is also what makes a recursive macro terminate — a base
644+case sits above the recursive rule. Recursion falls out for free: an expansion
645+containing another macro call is lowered like any expression, so it expands
646+again, with a 64-level limit so a non-terminating macro is an error rather
647+than a hang.
644648
645649 A definition it cannot handle is recorded *with its reason*, so a call site
646650 says "`foo!` cannot be expanded: nested `$(..)` repetition is not implemented
647651 yet" rather than "unknown macro". Captured fragments are parenthesised on
648652 substitution, so `square!(2 + 3)` is 25 and not 11.
649653
650-Repetition needed two things that are easy to get wrong, both caught by
651-`tests/cases/038`:
654+Three things here are easy to get wrong, all caught by `tests/cases/038` and
655+`039` rather than by reasoning:
652656
653657 - `>` closes `=>` and `->` as well as a generic argument list, so the nesting
654658 depth used to find a fragment's end must not go negative. Without that,
@@ -658,6 +662,11 @@ Repetition needed two things that are easy to get wrong, both caught by
658662 leads each iteration. A trailing fragment therefore stops at the separator
659663 when there is one and at whatever starts the next iteration when there is
660664 not.
665+- **A fragment never spans a top-level `,` or `;`.** Without that, a greedy
666+ `$x:expr` swallows the whole invocation and `describe!(3, 4)` matches the
667+ *one-argument* rule, silently taking the wrong branch. A comma inside
668+ brackets is within a `Group` token and so is not at this level at all,
669+ which is why the rule is safe.
661670
662671 A block expression with statements (`{{ let mut m = ..; m }}`, which is how
663672 these macros are usually written) now lowers to Nim's `block:` expression
@@ -668,10 +677,13 @@ not collide in one scope.
668677 of 400 crates accepted.** A `macro_rules!` used to be a hard stop at item
669678 level, failing a whole crate on sight.
670679
671-Repetition then moved it by **zero** — 18 before, 18 after — for the reason
672-every previous feature did: the crates it unblocked hit their next blocker.
673-That is now seven features running. The capability is real and tested; the
674-crate count is gated by something else.
680+Repetition and multi-rule then each moved it by **zero** — 18 before, 18
681+after, 12 compilable throughout — for the reason every previous feature did:
682+the crates they unblocked hit their next blocker. That is now eight features
683+running. The capabilities are real and tested; the crate count is gated by
684+something else, and the remaining blockers say what: unsupported types (86)
685+and calls into dependencies (49), which is the unbounded tail rather than the
686+finite feature pool.
675687
676688 ### A stricter number
677689
@@ -630,25 +630,29 @@ which is exactly the guessing the project refuses. Expanding at the call site
630 yields ordinary Rust in a context where the types are known, so it lowers like630 yields ordinary Rust in a context where the types are known, so it lowers like
631 anything else. Same applicability, faithful output.631 anything else. Same applicability, faithful output.
632 632
633-`src/mrules.rs` implements single-rule macros, **with `$(..)` repetition**.633+`src/mrules.rs` handles any number of rules, with `$(..)` repetition.
634-Recounting the 1,833 definitions by the expander's actual boundary — one rule,634+Measured against the 1,833 definitions in the sample:
635-any repetition, no `:tt`:
636 635
637-| | count |636+| | count | |
638-|---|---|637+|---|---|---|
639-| single rule, no repetition | 661 |638+| **expandable** | **1,369** | **74%** |
640-| single rule, with repetition | 271 |639+| nested `$(..)` inside `$(..)` | 44 | 2% |
641-| **expandable** | **932 (51%)** |640+| `:tt` token-tree munching | 420 | 22% |
642-| multiple rules | 481 |641+
643-| `:tt` | 420 |642+Rules are tried top to bottom and the first whose matcher matches wins, which
643+is Rust's rule and is also what makes a recursive macro terminate — a base
644+case sits above the recursive rule. Recursion falls out for free: an expansion
645+containing another macro call is lowered like any expression, so it expands
646+again, with a 64-level limit so a non-terminating macro is an error rather
647+than a hang.
644 648
645 A definition it cannot handle is recorded *with its reason*, so a call site649 A definition it cannot handle is recorded *with its reason*, so a call site
646 says "`foo!` cannot be expanded: nested `$(..)` repetition is not implemented650 says "`foo!` cannot be expanded: nested `$(..)` repetition is not implemented
647 yet" rather than "unknown macro". Captured fragments are parenthesised on651 yet" rather than "unknown macro". Captured fragments are parenthesised on
648 substitution, so `square!(2 + 3)` is 25 and not 11.652 substitution, so `square!(2 + 3)` is 25 and not 11.
649 653
650-Repetition needed two things that are easy to get wrong, both caught by654+Three things here are easy to get wrong, all caught by `tests/cases/038` and
651-`tests/cases/038`:655+`039` rather than by reasoning:
652 656
653 - `>` closes `=>` and `->` as well as a generic argument list, so the nesting657 - `>` closes `=>` and `->` as well as a generic argument list, so the nesting
654 depth used to find a fragment's end must not go negative. Without that,658 depth used to find a fragment's end must not go negative. Without that,
@@ -658,6 +662,11 @@ Repetition needed two things that are easy to get wrong, both caught by
658 leads each iteration. A trailing fragment therefore stops at the separator662 leads each iteration. A trailing fragment therefore stops at the separator
659 when there is one and at whatever starts the next iteration when there is663 when there is one and at whatever starts the next iteration when there is
660 not.664 not.
665+- **A fragment never spans a top-level `,` or `;`.** Without that, a greedy
666+ `$x:expr` swallows the whole invocation and `describe!(3, 4)` matches the
667+ *one-argument* rule, silently taking the wrong branch. A comma inside
668+ brackets is within a `Group` token and so is not at this level at all,
669+ which is why the rule is safe.
661 670
662 A block expression with statements (`{{ let mut m = ..; m }}`, which is how671 A block expression with statements (`{{ let mut m = ..; m }}`, which is how
663 these macros are usually written) now lowers to Nim's `block:` expression672 these macros are usually written) now lowers to Nim's `block:` expression
@@ -668,10 +677,13 @@ not collide in one scope.
668 of 400 crates accepted.** A `macro_rules!` used to be a hard stop at item677 of 400 crates accepted.** A `macro_rules!` used to be a hard stop at item
669 level, failing a whole crate on sight.678 level, failing a whole crate on sight.
670 679
671-Repetition then moved it by **zero** — 18 before, 18 after — for the reason680+Repetition and multi-rule then each moved it by **zero** — 18 before, 18
672-every previous feature did: the crates it unblocked hit their next blocker.681+after, 12 compilable throughout — for the reason every previous feature did:
673-That is now seven features running. The capability is real and tested; the682+the crates they unblocked hit their next blocker. That is now eight features
674-crate count is gated by something else.683+running. The capabilities are real and tested; the crate count is gated by
684+something else, and the remaining blockers say what: unsupported types (86)
685+and calls into dependencies (49), which is the unbounded tail rather than the
686+finite feature pool.
675 687
676 ### A stricter number688 ### A stricter number
677 689
modified README.md +3 -3
@@ -111,9 +111,9 @@ A `macro_rules!` is **expanded at the call site**, not translated into a Nim
111111 template — but a template body is untyped, and this lowering is type-directed
112112 throughout. Expanding gives ordinary Rust where the types are known.
113113
114-Single-rule macros are supported, with `$(..)` repetition — 932 of the 1,833
115-definitions in that sample, 51%. The rest are multi-rule (481) or token-tree
116-munchers (420).
114+Any number of rules, with `$(..)` repetition and recursion — 1,369 of the
115+1,833 definitions in that sample, 74%. What remains is token-tree munching
116+(420) and nested repetition (44).
117117
118118 Expansion took the survey from 3 to 18 of 400 crates accepted, of which 12
119119 produce Nim the Nim compiler accepts. (Accepted, compiles, and
@@ -111,9 +111,9 @@ A `macro_rules!` is **expanded at the call site**, not translated into a Nim
111 template — but a template body is untyped, and this lowering is type-directed111 template — but a template body is untyped, and this lowering is type-directed
112 throughout. Expanding gives ordinary Rust where the types are known.112 throughout. Expanding gives ordinary Rust where the types are known.
113 113
114-Single-rule macros are supported, with `$(..)` repetition — 932 of the 1,833114+Any number of rules, with `$(..)` repetition and recursion — 1,369 of the
115-definitions in that sample, 51%. The rest are multi-rule (481) or token-tree115+1,833 definitions in that sample, 74%. What remains is token-tree munching
116-munchers (420).116+(420) and nested repetition (44).
117 117
118 Expansion took the survey from 3 to 18 of 400 crates accepted, of which 12118 Expansion took the survey from 3 to 18 of 400 crates accepted, of which 12
119 produce Nim the Nim compiler accepts. (Accepted, compiles, and119 produce Nim the Nim compiler accepts. (Accepted, compiles, and
modified src/lower.rs +15 -1
@@ -224,6 +224,7 @@ pub struct Lowerer {
224224 /// cannot, so a call site can say *why* rather than "unknown macro".
225225 mrules: HashMap<String, crate::mrules::MacroDef>,
226226 mrules_bad: HashMap<String, String>,
227+ macro_depth: usize,
227228 /// Types declared by a `bitflags!` invocation.
228229 bitflags: std::collections::HashSet<String>,
229230 /// `(type, flag) -> nim const name`.
@@ -307,6 +308,7 @@ impl Lowerer {
307308 const_ptrs: std::collections::HashSet::new(),
308309 mrules: HashMap::new(),
309310 mrules_bad: HashMap::new(),
311+ macro_depth: 0,
310312 bitflags: std::collections::HashSet::new(),
311313 flag_consts: HashMap::new(),
312314 use_map: HashMap::new(),
@@ -5086,6 +5088,15 @@ impl Lowerer {
50865088 return Err(format!("`{other}!` cannot be expanded: {why}"));
50875089 }
50885090 if let Some(def) = self.mrules.get(other).cloned() {
5091+ // A multi-rule macro commonly recurses, shedding one
5092+ // argument per step. The depth limit turns a macro that
5093+ // does not terminate into an error rather than a hang.
5094+ if self.macro_depth > 64 {
5095+ return Err(format!(
5096+ "`{other}!` expanded more than 64 levels deep; it \
5097+ does not appear to terminate"
5098+ ));
5099+ }
50895100 let expanded = def
50905101 .expand(mac.tokens.clone())
50915102 .map_err(|e| format!("expanding `{other}!`: {e}"))?;
@@ -5098,7 +5109,10 @@ impl Lowerer {
50985109 expanded
50995110 )
51005111 })?;
5101- return Ok(self.expr(&e)?.code);
5112+ self.macro_depth += 1;
5113+ let r = self.expr(&e);
5114+ self.macro_depth -= 1;
5115+ return Ok(r?.code);
51025116 }
51035117 Err(format!(
51045118 "unsupported macro `{other}!`; a macro whose expansion is not \
@@ -224,6 +224,7 @@ pub struct Lowerer {
224 /// cannot, so a call site can say *why* rather than "unknown macro".224 /// cannot, so a call site can say *why* rather than "unknown macro".
225 mrules: HashMap<String, crate::mrules::MacroDef>,225 mrules: HashMap<String, crate::mrules::MacroDef>,
226 mrules_bad: HashMap<String, String>,226 mrules_bad: HashMap<String, String>,
227+ macro_depth: usize,
227 /// Types declared by a `bitflags!` invocation.228 /// Types declared by a `bitflags!` invocation.
228 bitflags: std::collections::HashSet<String>,229 bitflags: std::collections::HashSet<String>,
229 /// `(type, flag) -> nim const name`.230 /// `(type, flag) -> nim const name`.
@@ -307,6 +308,7 @@ impl Lowerer {
307 const_ptrs: std::collections::HashSet::new(),308 const_ptrs: std::collections::HashSet::new(),
308 mrules: HashMap::new(),309 mrules: HashMap::new(),
309 mrules_bad: HashMap::new(),310 mrules_bad: HashMap::new(),
311+ macro_depth: 0,
310 bitflags: std::collections::HashSet::new(),312 bitflags: std::collections::HashSet::new(),
311 flag_consts: HashMap::new(),313 flag_consts: HashMap::new(),
312 use_map: HashMap::new(),314 use_map: HashMap::new(),
@@ -5086,6 +5088,15 @@ impl Lowerer {
5086 return Err(format!("`{other}!` cannot be expanded: {why}"));5088 return Err(format!("`{other}!` cannot be expanded: {why}"));
5087 }5089 }
5088 if let Some(def) = self.mrules.get(other).cloned() {5090 if let Some(def) = self.mrules.get(other).cloned() {
5091+ // A multi-rule macro commonly recurses, shedding one
5092+ // argument per step. The depth limit turns a macro that
5093+ // does not terminate into an error rather than a hang.
5094+ if self.macro_depth > 64 {
5095+ return Err(format!(
5096+ "`{other}!` expanded more than 64 levels deep; it \
5097+ does not appear to terminate"
5098+ ));
5099+ }
5089 let expanded = def5100 let expanded = def
5090 .expand(mac.tokens.clone())5101 .expand(mac.tokens.clone())
5091 .map_err(|e| format!("expanding `{other}!`: {e}"))?;5102 .map_err(|e| format!("expanding `{other}!`: {e}"))?;
@@ -5098,7 +5109,10 @@ impl Lowerer {
5098 expanded5109 expanded
5099 )5110 )
5100 })?;5111 })?;
5101- return Ok(self.expr(&e)?.code);5112+ self.macro_depth += 1;
5113+ let r = self.expr(&e);
5114+ self.macro_depth -= 1;
5115+ return Ok(r?.code);
5102 }5116 }
5103 Err(format!(5117 Err(format!(
5104 "unsupported macro `{other}!`; a macro whose expansion is not \5118 "unsupported macro `{other}!`; a macro whose expansion is not \
modified src/mrules.rs +62 -25
@@ -42,31 +42,48 @@ enum Cap {
4242 type Binds = HashMap<String, Cap>;
4343
4444 #[derive(Debug, Clone)]
45-pub struct MacroDef {
45+struct Rule {
4646 pattern: Vec<Pat>,
4747 body: TokenStream,
4848 }
4949
50-/// Parse `macro_rules!`'s body: `( $matcher ) => { $transcriber };`.
50+#[derive(Debug, Clone)]
51+pub struct MacroDef {
52+ /// Rules in source order. Rust tries them top to bottom and the first
53+ /// whose matcher matches wins, so order is semantics, not style.
54+ rules: Vec<Rule>,
55+}
56+
57+/// Parse `macro_rules!`'s body: a `;`-separated list of
58+/// `( $matcher ) => { $transcriber }`.
5159 pub fn parse(tokens: TokenStream) -> Result<MacroDef, String> {
5260 let t: Vec<TokenTree> = tokens.into_iter().collect();
53- // matcher group, `=`, `>`, transcriber group, optional `;`
54- let (matcher, rest) = match t.split_first() {
55- Some((TokenTree::Group(g), rest)) => (g.clone(), rest),
56- _ => return Err("expected a matcher group".into()),
57- };
58- let arrow: String = rest.iter().take(2).map(|t| t.to_string()).collect();
59- if arrow != "=>" {
60- return Err("expected `=>`".into());
61+ let mut rules = Vec::new();
62+ let mut i = 0;
63+ while i < t.len() {
64+ let matcher = match &t[i] {
65+ TokenTree::Group(g) => g.clone(),
66+ other => return Err(format!("expected a matcher group, found `{other}`")),
67+ };
68+ let arrow: String = t[i + 1..].iter().take(2).map(|t| t.to_string()).collect();
69+ if arrow != "=>" {
70+ return Err("expected `=>` after a matcher".into());
71+ }
72+ let body = match t.get(i + 3) {
73+ Some(TokenTree::Group(g)) => g.stream(),
74+ _ => return Err("expected a transcriber group".into()),
75+ };
76+ rules.push(Rule { pattern: parse_pattern(matcher.stream())?, body });
77+ i += 4;
78+ // The separator is optional after the last rule.
79+ if i < t.len() && t[i].to_string() == ";" {
80+ i += 1;
81+ }
6182 }
62- let body = match rest.get(2) {
63- Some(TokenTree::Group(g)) => g.stream(),
64- _ => return Err("expected a transcriber group".into()),
65- };
66- if rest.len() > 4 || (rest.len() == 4 && rest[3].to_string() != ";") {
67- return Err("more than one rule is not implemented yet".into());
83+ if rules.is_empty() {
84+ return Err("a macro with no rules".into());
6885 }
69- Ok(MacroDef { pattern: parse_pattern(matcher.stream())?, body })
86+ Ok(MacroDef { rules })
7087 }
7188
7289 fn parse_pattern(ts: TokenStream) -> Result<Vec<Pat>, String> {
@@ -136,15 +153,25 @@ fn parse_pattern(ts: TokenStream) -> Result<Vec<Pat>, String> {
136153 }
137154
138155 impl MacroDef {
139- /// Match an invocation's tokens and substitute them into the body.
156+ /// Try each rule in order and substitute using the first that matches,
157+ /// which is what Rust does.
140158 pub fn expand(&self, input: TokenStream) -> Result<TokenStream, String> {
141- let mut binds = Binds::new();
142159 let toks: Vec<TokenTree> = input.into_iter().collect();
143- let used = match_seq(&self.pattern, &toks, &mut binds, None)?;
144- if used != toks.len() {
145- return Err("the invocation has tokens the matcher does not consume".into());
160+ let mut why = Vec::new();
161+ for (n, rule) in self.rules.iter().enumerate() {
162+ let mut binds = Binds::new();
163+ match match_seq(&rule.pattern, &toks, &mut binds, None) {
164+ Ok(used) if used == toks.len() => {
165+ return Ok(substitute(rule.body.clone(), &binds))
166+ }
167+ Ok(_) => why.push(format!(
168+ "rule {}: matched, but left tokens over",
169+ n + 1
170+ )),
171+ Err(e) => why.push(format!("rule {}: {e}", n + 1)),
172+ }
146173 }
147- Ok(substitute(self.body.clone(), &binds))
174+ Err(format!("no rule matched ({})", why.join("; ")))
148175 }
149176 }
150177
@@ -241,8 +268,18 @@ fn match_seq(
241268 let mut depth = 0i32;
242269 while i < toks.len() {
243270 let s = toks[i].to_string();
244- if let Some(stop) = &stop {
245- if depth == 0 && s == *stop {
271+ if depth == 0 {
272+ if let Some(stop) = &stop {
273+ if s == *stop {
274+ break;
275+ }
276+ }
277+ // A fragment never spans a top-level `,` or `;`: an
278+ // `expr` is one expression, and a comma at this level
279+ // separates arguments rather than belonging to one.
280+ // A comma inside brackets is within a `Group` token,
281+ // so it is not at this level at all.
282+ if s == "," || s == ";" {
246283 break;
247284 }
248285 }
@@ -42,31 +42,48 @@ enum Cap {
42 type Binds = HashMap<String, Cap>;42 type Binds = HashMap<String, Cap>;
43 43
44 #[derive(Debug, Clone)]44 #[derive(Debug, Clone)]
45-pub struct MacroDef {45+struct Rule {
46 pattern: Vec<Pat>,46 pattern: Vec<Pat>,
47 body: TokenStream,47 body: TokenStream,
48 }48 }
49 49
50-/// Parse `macro_rules!`'s body: `( $matcher ) => { $transcriber };`.50+#[derive(Debug, Clone)]
51+pub struct MacroDef {
52+ /// Rules in source order. Rust tries them top to bottom and the first
53+ /// whose matcher matches wins, so order is semantics, not style.
54+ rules: Vec<Rule>,
55+}
56+
57+/// Parse `macro_rules!`'s body: a `;`-separated list of
58+/// `( $matcher ) => { $transcriber }`.
51 pub fn parse(tokens: TokenStream) -> Result<MacroDef, String> {59 pub fn parse(tokens: TokenStream) -> Result<MacroDef, String> {
52 let t: Vec<TokenTree> = tokens.into_iter().collect();60 let t: Vec<TokenTree> = tokens.into_iter().collect();
53- // matcher group, `=`, `>`, transcriber group, optional `;`61+ let mut rules = Vec::new();
54- let (matcher, rest) = match t.split_first() {62+ let mut i = 0;
55- Some((TokenTree::Group(g), rest)) => (g.clone(), rest),63+ while i < t.len() {
56- _ => return Err("expected a matcher group".into()),64+ let matcher = match &t[i] {
57- };65+ TokenTree::Group(g) => g.clone(),
58- let arrow: String = rest.iter().take(2).map(|t| t.to_string()).collect();66+ other => return Err(format!("expected a matcher group, found `{other}`")),
59- if arrow != "=>" {67+ };
60- return Err("expected `=>`".into());68+ let arrow: String = t[i + 1..].iter().take(2).map(|t| t.to_string()).collect();
69+ if arrow != "=>" {
70+ return Err("expected `=>` after a matcher".into());
71+ }
72+ let body = match t.get(i + 3) {
73+ Some(TokenTree::Group(g)) => g.stream(),
74+ _ => return Err("expected a transcriber group".into()),
75+ };
76+ rules.push(Rule { pattern: parse_pattern(matcher.stream())?, body });
77+ i += 4;
78+ // The separator is optional after the last rule.
79+ if i < t.len() && t[i].to_string() == ";" {
80+ i += 1;
81+ }
61 }82 }
62- let body = match rest.get(2) {83+ if rules.is_empty() {
63- Some(TokenTree::Group(g)) => g.stream(),84+ return Err("a macro with no rules".into());
64- _ => return Err("expected a transcriber group".into()),
65- };
66- if rest.len() > 4 || (rest.len() == 4 && rest[3].to_string() != ";") {
67- return Err("more than one rule is not implemented yet".into());
68 }85 }
69- Ok(MacroDef { pattern: parse_pattern(matcher.stream())?, body })86+ Ok(MacroDef { rules })
70 }87 }
71 88
72 fn parse_pattern(ts: TokenStream) -> Result<Vec<Pat>, String> {89 fn parse_pattern(ts: TokenStream) -> Result<Vec<Pat>, String> {
@@ -136,15 +153,25 @@ fn parse_pattern(ts: TokenStream) -> Result<Vec<Pat>, String> {
136 }153 }
137 154
138 impl MacroDef {155 impl MacroDef {
139- /// Match an invocation's tokens and substitute them into the body.156+ /// Try each rule in order and substitute using the first that matches,
157+ /// which is what Rust does.
140 pub fn expand(&self, input: TokenStream) -> Result<TokenStream, String> {158 pub fn expand(&self, input: TokenStream) -> Result<TokenStream, String> {
141- let mut binds = Binds::new();
142 let toks: Vec<TokenTree> = input.into_iter().collect();159 let toks: Vec<TokenTree> = input.into_iter().collect();
143- let used = match_seq(&self.pattern, &toks, &mut binds, None)?;160+ let mut why = Vec::new();
144- if used != toks.len() {161+ for (n, rule) in self.rules.iter().enumerate() {
145- return Err("the invocation has tokens the matcher does not consume".into());162+ let mut binds = Binds::new();
163+ match match_seq(&rule.pattern, &toks, &mut binds, None) {
164+ Ok(used) if used == toks.len() => {
165+ return Ok(substitute(rule.body.clone(), &binds))
166+ }
167+ Ok(_) => why.push(format!(
168+ "rule {}: matched, but left tokens over",
169+ n + 1
170+ )),
171+ Err(e) => why.push(format!("rule {}: {e}", n + 1)),
172+ }
146 }173 }
147- Ok(substitute(self.body.clone(), &binds))174+ Err(format!("no rule matched ({})", why.join("; ")))
148 }175 }
149 }176 }
150 177
@@ -241,8 +268,18 @@ fn match_seq(
241 let mut depth = 0i32;268 let mut depth = 0i32;
242 while i < toks.len() {269 while i < toks.len() {
243 let s = toks[i].to_string();270 let s = toks[i].to_string();
244- if let Some(stop) = &stop {271+ if depth == 0 {
245- if depth == 0 && s == *stop {272+ if let Some(stop) = &stop {
273+ if s == *stop {
274+ break;
275+ }
276+ }
277+ // A fragment never spans a top-level `,` or `;`: an
278+ // `expr` is one expression, and a comma at this level
279+ // separates arguments rather than belonging to one.
280+ // A comma inside brackets is within a `Group` token,
281+ // so it is not at this level at all.
282+ if s == "," || s == ";" {
246 break;283 break;
247 }284 }
248 }285 }
added tests/cases/039-macro-multi-rule.rs +44 -0
new file mode 100644
@@ -0,0 +1,44 @@
1+// Rules are tried top to bottom and the first whose matcher matches wins, so
2+// their order is semantics rather than style. That is also what makes a
3+// recursive macro terminate: a base-case rule sits above the recursive one.
4+
5+macro_rules! describe {
6+ () => { 0 };
7+ ($x:expr) => { $x };
8+ ($x:expr, $y:expr) => { $x * 100 + $y };
9+}
10+
11+// Recursion, shedding one argument per step.
12+macro_rules! total {
13+ () => { 0 };
14+ ($x:expr) => { $x };
15+ ($x:expr, $($rest:expr),+) => { $x + total!($($rest),+) };
16+}
17+
18+macro_rules! depth {
19+ ($x:expr) => { 1 };
20+ ($x:expr, $($rest:expr),+) => { 1 + depth!($($rest),+) };
21+}
22+
23+// Different shapes, not just different arities.
24+macro_rules! pick {
25+ (first $a:expr, $b:expr) => { $a };
26+ (second $a:expr, $b:expr) => { $b };
27+ (sum $a:expr, $b:expr) => { $a + $b };
28+}
29+
30+fn main() {
31+ println!("{} {} {}", describe!(), describe!(7), describe!(3, 4));
32+
33+ println!("{}", total!());
34+ println!("{}", total!(5));
35+ println!("{}", total!(1, 2, 3, 4, 5));
36+ println!("{}", total!(2 * 3, 4 + 1));
37+
38+ println!("{} {}", depth!(9), depth!(9, 9, 9, 9));
39+
40+ println!("{} {} {}", pick!(first 10, 20), pick!(second 10, 20), pick!(sum 10, 20));
41+
42+ let a: i64 = 1000;
43+ println!("{}", total!(a, a, a));
44+}
new file mode 100644
@@ -0,0 +1,44 @@
1+// Rules are tried top to bottom and the first whose matcher matches wins, so
2+// their order is semantics rather than style. That is also what makes a
3+// recursive macro terminate: a base-case rule sits above the recursive one.
4+
5+macro_rules! describe {
6+ () => { 0 };
7+ ($x:expr) => { $x };
8+ ($x:expr, $y:expr) => { $x * 100 + $y };
9+}
10+
11+// Recursion, shedding one argument per step.
12+macro_rules! total {
13+ () => { 0 };
14+ ($x:expr) => { $x };
15+ ($x:expr, $($rest:expr),+) => { $x + total!($($rest),+) };
16+}
17+
18+macro_rules! depth {
19+ ($x:expr) => { 1 };
20+ ($x:expr, $($rest:expr),+) => { 1 + depth!($($rest),+) };
21+}
22+
23+// Different shapes, not just different arities.
24+macro_rules! pick {
25+ (first $a:expr, $b:expr) => { $a };
26+ (second $a:expr, $b:expr) => { $b };
27+ (sum $a:expr, $b:expr) => { $a + $b };
28+}
29+
30+fn main() {
31+ println!("{} {} {}", describe!(), describe!(7), describe!(3, 4));
32+
33+ println!("{}", total!());
34+ println!("{}", total!(5));
35+ println!("{}", total!(1, 2, 3, 4, 5));
36+ println!("{}", total!(2 * 3, 4 + 1));
37+
38+ println!("{} {}", depth!(9), depth!(9, 9, 9, 9));
39+
40+ println!("{} {} {}", pick!(first 10, 20), pick!(second 10, 20), pick!(sum 10, 20));
41+
42+ let a: i64 = 1000;
43+ println!("{}", total!(a, a, a));
44+}