Add `--expand`: run proc macros at ingestion, so the output is pure Nim
I had called proc macros a floor -- compile-time programs with no Rust to
translate. That is true of the macro and false of the conclusion. The macro
cannot be translated, but it can be run, and what it emits is ordinary Rust.
Running it at ingestion means the Nim that comes out needs no Rust toolchain
behind it; rustc is needed where it already was, when rustnim runs.
`--expand` does that: rustc's -Zunpretty=expanded over the input, then lower
the result. tests/cases/047 is transpiled from expanded source and exercises
what #[derive(..)] actually generates, with output identical to rustc's.
Three things were needed to lower real expansions, and each is general rather
than derive-specific. UFCS calls: `Trait::method(x, ..)` is `x.method(..)`,
which expansions lean on because a generated `Clone::clone(&self.x)` needs
nothing in scope. The Debug derive's `Formatter::debug_struct_fieldN_finish`,
which renders `Point { x: 1, y: 2 }` -- built directly rather than calling a
std helper we do not have. And `format_args!` with `_print`/`_eprint`, which
is what println! desugars to once macros are expanded.
Also `let _: AssertParamIsClone<i32>;`, the Copy derive's compile-time
assertion, which has nothing to run.
DESIGN.md's "floor" section is corrected. Neither proc macros nor FFI is a
floor: macros run at ingestion, and a crate calling libc needs its calls
bound rather than libc transpiled, which extern "C" -> importc already does.
Intrinsics are the same story one level down. What is left is not a wall but
a quantity -- 741 crates, each needing its stack cleared and each mapping
verified -- which is a much less interesting claim than "it cannot be done",
and the one the evidence supports.
53 differential cases, all green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>e7dbe75 parent: effd8a3 modified
DESIGN.md +31 -11 | @@ -435,21 +435,41 @@ problem 741 times, not a different one. | ||
| 435 | 435 | **The floor, which is not Rust at all.** The graph terminates in things that |
| 436 | 436 | have no Rust source to translate: |
| 437 | 437 | |
| 438 | -- **Proc macros** — 37 of the 400 sampled crates are `proc-macro = true`. | |
| 439 | - These are *compile-time programs*: `syn`, `quote` and `proc-macro2` have 155 | |
| 440 | - dependents between them in `libcosmic`'s tree. They bottom out in | |
| 441 | - `proc_macro::TokenStream`, which the compiler provides and which exists as | |
| 442 | - no crate in the registry. There is nothing to transpile — they have to be | |
| 443 | - *run*, and what they emit is the Rust you would then translate. | |
| 438 | +- **Proc macros** — 37 of the 400 sampled crates are `proc-macro = true`, and | |
| 439 | + `syn`, `quote` and `proc-macro2` have 155 dependents between them in | |
| 440 | + `libcosmic`'s tree. A proc macro is a compile-time *program*: it cannot be | |
| 441 | + translated, only run. | |
| 444 | 442 | - **FFI** — `libc` is 129,594 lines holding 121 function bodies; the rest is |
| 445 | - declarations binding to the C library. `wgpu`, `winit` and the `*-sys` | |
| 446 | - crates are the same shape. | |
| 443 | + declarations binding to the C library. | |
| 447 | 444 | - **Intrinsics** — SIMD and the parts of `core` the compiler implements |
| 448 | 445 | directly. |
| 449 | 446 | |
| 450 | -That is the real answer to "isn't it just Rust?": yes, all the way down until | |
| 451 | -it stops being Rust, and the stopping points are where the interesting | |
| 452 | -dependencies live. | |
| 447 | +### Neither of the first two is actually a floor | |
| 448 | + | |
| 449 | +Both were called one in an earlier draft, and both turn out not to be. | |
| 450 | + | |
| 451 | +**Proc macros run at ingestion, not in the output.** `--expand` runs rustc's | |
| 452 | +macro expansion before lowering, so what is transpiled is what the macro | |
| 453 | +generated — ordinary Rust. rustc is needed when rustnim runs, which it already | |
| 454 | +is; what rustnim *writes* is Nim with no Rust toolchain behind it. | |
| 455 | +`tests/cases/047` is transpiled from expanded source and exercises what | |
| 456 | +`#[derive(..)]` really emits: UFCS calls (`Clone::clone(&self.x)`), the | |
| 457 | +`Formatter::debug_struct_fieldN_finish` helper, and `println!` desugared to | |
| 458 | +`_print(format_args!(..))`. Output identical to rustc's, `Point { x: 1, y: -2 }` | |
| 459 | +and all. | |
| 460 | + | |
| 461 | +**FFI is bound, not translated.** A crate calling `libc` does not need `libc` | |
| 462 | +transpiled; it needs those calls bound, and `extern "C"` already lowers to Nim | |
| 463 | +`importc`. Nim reaches the same symbols — they are the same objects. | |
| 464 | + | |
| 465 | +Intrinsics are the same story one level down: SIMD builtins are reachable | |
| 466 | +through C headers by the same `importc` route. | |
| 467 | + | |
| 468 | +So the honest answer to "isn't it just Rust?" is **yes, and the places it | |
| 469 | +stops being Rust are reachable by other means**. What is left is not a wall | |
| 470 | +but a quantity: 741 crates, each needing its stack cleared and each mapping | |
| 471 | +verified. That is a different and much less interesting claim than "it cannot | |
| 472 | +be done", and it is the one the evidence supports. | |
| 453 | 473 | |
| 454 | 474 | ### The blocker survey, and what it says about roadmaps |
| 455 | 475 | |
| @@ -435,21 +435,41 @@ problem 741 times, not a different one. | |||
| 435 | **The floor, which is not Rust at all.** The graph terminates in things that | 435 | **The floor, which is not Rust at all.** The graph terminates in things that |
| 436 | have no Rust source to translate: | 436 | have no Rust source to translate: |
| 437 | 437 | ||
| 438 | -- **Proc macros** — 37 of the 400 sampled crates are `proc-macro = true`. | 438 | +- **Proc macros** — 37 of the 400 sampled crates are `proc-macro = true`, and |
| 439 | - These are *compile-time programs*: `syn`, `quote` and `proc-macro2` have 155 | 439 | + `syn`, `quote` and `proc-macro2` have 155 dependents between them in |
| 440 | - dependents between them in `libcosmic`'s tree. They bottom out in | 440 | + `libcosmic`'s tree. A proc macro is a compile-time *program*: it cannot be |
| 441 | - `proc_macro::TokenStream`, which the compiler provides and which exists as | 441 | + translated, only run. |
| 442 | - no crate in the registry. There is nothing to transpile — they have to be | ||
| 443 | - *run*, and what they emit is the Rust you would then translate. | ||
| 444 | - **FFI** — `libc` is 129,594 lines holding 121 function bodies; the rest is | 442 | - **FFI** — `libc` is 129,594 lines holding 121 function bodies; the rest is |
| 445 | - declarations binding to the C library. `wgpu`, `winit` and the `*-sys` | 443 | + declarations binding to the C library. |
| 446 | - crates are the same shape. | ||
| 447 | - **Intrinsics** — SIMD and the parts of `core` the compiler implements | 444 | - **Intrinsics** — SIMD and the parts of `core` the compiler implements |
| 448 | directly. | 445 | directly. |
| 449 | 446 | ||
| 450 | -That is the real answer to "isn't it just Rust?": yes, all the way down until | 447 | +### Neither of the first two is actually a floor |
| 451 | -it stops being Rust, and the stopping points are where the interesting | 448 | + |
| 452 | -dependencies live. | 449 | +Both were called one in an earlier draft, and both turn out not to be. |
| 450 | + | ||
| 451 | +**Proc macros run at ingestion, not in the output.** `--expand` runs rustc's | ||
| 452 | +macro expansion before lowering, so what is transpiled is what the macro | ||
| 453 | +generated — ordinary Rust. rustc is needed when rustnim runs, which it already | ||
| 454 | +is; what rustnim *writes* is Nim with no Rust toolchain behind it. | ||
| 455 | +`tests/cases/047` is transpiled from expanded source and exercises what | ||
| 456 | +`#[derive(..)]` really emits: UFCS calls (`Clone::clone(&self.x)`), the | ||
| 457 | +`Formatter::debug_struct_fieldN_finish` helper, and `println!` desugared to | ||
| 458 | +`_print(format_args!(..))`. Output identical to rustc's, `Point { x: 1, y: -2 }` | ||
| 459 | +and all. | ||
| 460 | + | ||
| 461 | +**FFI is bound, not translated.** A crate calling `libc` does not need `libc` | ||
| 462 | +transpiled; it needs those calls bound, and `extern "C"` already lowers to Nim | ||
| 463 | +`importc`. Nim reaches the same symbols — they are the same objects. | ||
| 464 | + | ||
| 465 | +Intrinsics are the same story one level down: SIMD builtins are reachable | ||
| 466 | +through C headers by the same `importc` route. | ||
| 467 | + | ||
| 468 | +So the honest answer to "isn't it just Rust?" is **yes, and the places it | ||
| 469 | +stops being Rust are reachable by other means**. What is left is not a wall | ||
| 470 | +but a quantity: 741 crates, each needing its stack cleared and each mapping | ||
| 471 | +verified. That is a different and much less interesting claim than "it cannot | ||
| 472 | +be done", and it is the one the evidence supports. | ||
| 453 | 473 | ||
| 454 | ### The blocker survey, and what it says about roadmaps | 474 | ### The blocker survey, and what it says about roadmaps |
| 455 | 475 | ||
modified
src/lower.rs +92 -1 | @@ -2114,6 +2114,16 @@ impl Lowerer { | ||
| 2114 | 2114 | } |
| 2115 | 2115 | } |
| 2116 | 2116 | |
| 2117 | + /// A string produced inside a formatting impl is *written*, not returned: | |
| 2118 | + /// the proc accumulates into `result`. Outside one it is just a string. | |
| 2119 | + fn fmt_value(&self, code: String) -> Val { | |
| 2120 | + if self.fmt_param.is_some() { | |
| 2121 | + Val::new(format!("result.add({})", code), Some(Nim::Unit)) | |
| 2122 | + } else { | |
| 2123 | + Val::new(code, Some(Nim::Prim("string".into()))) | |
| 2124 | + } | |
| 2125 | + } | |
| 2126 | + | |
| 2117 | 2127 | /// Whether an expression is the `Formatter` parameter of the formatting |
| 2118 | 2128 | /// impl currently being lowered. |
| 2119 | 2129 | fn is_fmt_param(&self, e: &Expr) -> bool { |
| @@ -2481,6 +2491,10 @@ impl Lowerer { | ||
| 2481 | 2491 | Pat::Ident(i) => (i.ident.to_string(), i.mutability.is_some(), None), |
| 2482 | 2492 | Pat::Type(t) => match &*t.pat { |
| 2483 | 2493 | Pat::Ident(i) => (i.ident.to_string(), i.mutability.is_some(), Some(self.map_ty(&t.ty)?)), |
| 2494 | + // `let _: AssertParamIsClone<i32>;` -- the `Copy` derive emits | |
| 2495 | + // these as a compile-time assertion. Rust checks a bound with | |
| 2496 | + // it; there is nothing to run. | |
| 2497 | + Pat::Wild(_) if l.init.is_none() => return Ok(()), | |
| 2484 | 2498 | _ => return Err("only `let <ident>` bindings are supported".into()), |
| 2485 | 2499 | }, |
| 2486 | 2500 | Pat::Wild(_) => ("_".into(), false, None), |
| @@ -5225,6 +5239,79 @@ impl Lowerer { | ||
| 5225 | 5239 | .as_ref() |
| 5226 | 5240 | .and_then(|k| self.fns.get(k)) |
| 5227 | 5241 | .map(|sig| Self::instantiate(sig, &arg_tys)); |
| 5242 | + // `println!` desugars to `_print(format_args!(..))`. The newline is | |
| 5243 | + // part of the format string by then, so this writes rather than echoes. | |
| 5244 | + if (name == "_print" || name == "_eprint") && codes.len() == 1 { | |
| 5245 | + let sink = if name == "_print" { "stdout" } else { "stderr" }; | |
| 5246 | + return Ok(Val::new( | |
| 5247 | + format!("{}.write({})", sink, codes[0]), | |
| 5248 | + Some(Nim::Unit), | |
| 5249 | + )); | |
| 5250 | + } | |
| 5251 | + | |
| 5252 | + // The `Debug` derive expands to `Formatter::debug_struct_fieldN_finish | |
| 5253 | + // (f, "Point", "x", &self.x, ..)`. Rust renders that as | |
| 5254 | + // `Point { x: 1, y: 2 }`, which is what is built here rather than | |
| 5255 | + // calling a std helper we do not have. | |
| 5256 | + if name.starts_with("debug_struct_field") && name.ends_with("_finish") | |
| 5257 | + || name == "debug_struct_fields_finish" | |
| 5258 | + { | |
| 5259 | + if codes.len() >= 2 && (codes.len() - 2) % 2 == 0 { | |
| 5260 | + let mut parts = vec![format!("{} & \" {{ \"", codes[1])]; | |
| 5261 | + let mut first = true; | |
| 5262 | + for pair in codes[2..].chunks(2) { | |
| 5263 | + if !first { | |
| 5264 | + parts.push("\", \"".to_string()); | |
| 5265 | + } | |
| 5266 | + first = false; | |
| 5267 | + parts.push(format!("{} & \": \" & rsDebug({})", pair[0], pair[1])); | |
| 5268 | + } | |
| 5269 | + parts.push("\" }\"".to_string()); | |
| 5270 | + return Ok(self.fmt_value(format!("({})", parts.join(" & ")))); | |
| 5271 | + } | |
| 5272 | + } | |
| 5273 | + // The tuple-struct form: `Point(1, 2)`. | |
| 5274 | + if name.starts_with("debug_tuple_field") && name.ends_with("_finish") { | |
| 5275 | + if codes.len() >= 2 { | |
| 5276 | + let inner: Vec<String> = codes[2..] | |
| 5277 | + .iter() | |
| 5278 | + .map(|c| format!("rsDebug({})", c)) | |
| 5279 | + .collect(); | |
| 5280 | + return Ok(self.fmt_value(format!( | |
| 5281 | + "({} & \"(\" & {} & \")\")", | |
| 5282 | + codes[1], | |
| 5283 | + inner.join(" & \", \" & ") | |
| 5284 | + ))); | |
| 5285 | + } | |
| 5286 | + } | |
| 5287 | + | |
| 5288 | + // `Trait::method(x, ..)` is the same call as `x.method(..)` -- Rust's | |
| 5289 | + // universal call syntax. Macro expansions lean on it heavily: a | |
| 5290 | + // generated `Clone::clone(&self.x)` needs nothing in scope. | |
| 5291 | + if ret.is_none() | |
| 5292 | + && !self.structs.contains_key(&name) | |
| 5293 | + && !self.enums.contains_key(&name) | |
| 5294 | + && p.path.segments.len() >= 2 | |
| 5295 | + && !c.args.is_empty() | |
| 5296 | + { | |
| 5297 | + let mut rest = syn::punctuated::Punctuated::new(); | |
| 5298 | + for a in c.args.iter().skip(1) { | |
| 5299 | + rest.push(a.clone()); | |
| 5300 | + } | |
| 5301 | + let as_method = syn::ExprMethodCall { | |
| 5302 | + attrs: Vec::new(), | |
| 5303 | + receiver: Box::new(c.args[0].clone()), | |
| 5304 | + dot_token: Default::default(), | |
| 5305 | + method: syn::Ident::new(&name, proc_macro2::Span::call_site()), | |
| 5306 | + turbofish: None, | |
| 5307 | + paren_token: Default::default(), | |
| 5308 | + args: rest, | |
| 5309 | + }; | |
| 5310 | + if let Ok(v) = self.method(&as_method, expect) { | |
| 5311 | + return Ok(v); | |
| 5312 | + } | |
| 5313 | + } | |
| 5314 | + | |
| 5228 | 5315 | if ret.is_none() && !self.structs.contains_key(&name) && !self.enums.contains_key(&name) { |
| 5229 | 5316 | return Err(format!( |
| 5230 | 5317 | "call to unknown function `{name}`; only functions defined in \ |
| @@ -5874,7 +5961,11 @@ impl Lowerer { | ||
| 5874 | 5961 | (true, false) => format!("stderr.write({s})"), |
| 5875 | 5962 | }) |
| 5876 | 5963 | } |
| 5877 | - "format" => self.format_args(mac), | |
| 5964 | + "format" | "format_args" | "format_args_nl" => { | |
| 5965 | + // `format_args!` is what `println!` desugars to once macros are | |
| 5966 | + // expanded; the trailing newline is already in the string. | |
| 5967 | + self.format_args(mac) | |
| 5968 | + } | |
| 5878 | 5969 | "write" | "writeln" => { |
| 5879 | 5970 | // `write!(f, "..", ..)` inside a formatting impl: the first |
| 5880 | 5971 | // argument is the sink, the rest is an ordinary format call. |
| @@ -2114,6 +2114,16 @@ impl Lowerer { | |||
| 2114 | } | 2114 | } |
| 2115 | } | 2115 | } |
| 2116 | 2116 | ||
| 2117 | + /// A string produced inside a formatting impl is *written*, not returned: | ||
| 2118 | + /// the proc accumulates into `result`. Outside one it is just a string. | ||
| 2119 | + fn fmt_value(&self, code: String) -> Val { | ||
| 2120 | + if self.fmt_param.is_some() { | ||
| 2121 | + Val::new(format!("result.add({})", code), Some(Nim::Unit)) | ||
| 2122 | + } else { | ||
| 2123 | + Val::new(code, Some(Nim::Prim("string".into()))) | ||
| 2124 | + } | ||
| 2125 | + } | ||
| 2126 | + | ||
| 2117 | /// Whether an expression is the `Formatter` parameter of the formatting | 2127 | /// Whether an expression is the `Formatter` parameter of the formatting |
| 2118 | /// impl currently being lowered. | 2128 | /// impl currently being lowered. |
| 2119 | fn is_fmt_param(&self, e: &Expr) -> bool { | 2129 | fn is_fmt_param(&self, e: &Expr) -> bool { |
| @@ -2481,6 +2491,10 @@ impl Lowerer { | |||
| 2481 | Pat::Ident(i) => (i.ident.to_string(), i.mutability.is_some(), None), | 2491 | Pat::Ident(i) => (i.ident.to_string(), i.mutability.is_some(), None), |
| 2482 | Pat::Type(t) => match &*t.pat { | 2492 | Pat::Type(t) => match &*t.pat { |
| 2483 | Pat::Ident(i) => (i.ident.to_string(), i.mutability.is_some(), Some(self.map_ty(&t.ty)?)), | 2493 | Pat::Ident(i) => (i.ident.to_string(), i.mutability.is_some(), Some(self.map_ty(&t.ty)?)), |
| 2494 | + // `let _: AssertParamIsClone<i32>;` -- the `Copy` derive emits | ||
| 2495 | + // these as a compile-time assertion. Rust checks a bound with | ||
| 2496 | + // it; there is nothing to run. | ||
| 2497 | + Pat::Wild(_) if l.init.is_none() => return Ok(()), | ||
| 2484 | _ => return Err("only `let <ident>` bindings are supported".into()), | 2498 | _ => return Err("only `let <ident>` bindings are supported".into()), |
| 2485 | }, | 2499 | }, |
| 2486 | Pat::Wild(_) => ("_".into(), false, None), | 2500 | Pat::Wild(_) => ("_".into(), false, None), |
| @@ -5225,6 +5239,79 @@ impl Lowerer { | |||
| 5225 | .as_ref() | 5239 | .as_ref() |
| 5226 | .and_then(|k| self.fns.get(k)) | 5240 | .and_then(|k| self.fns.get(k)) |
| 5227 | .map(|sig| Self::instantiate(sig, &arg_tys)); | 5241 | .map(|sig| Self::instantiate(sig, &arg_tys)); |
| 5242 | + // `println!` desugars to `_print(format_args!(..))`. The newline is | ||
| 5243 | + // part of the format string by then, so this writes rather than echoes. | ||
| 5244 | + if (name == "_print" || name == "_eprint") && codes.len() == 1 { | ||
| 5245 | + let sink = if name == "_print" { "stdout" } else { "stderr" }; | ||
| 5246 | + return Ok(Val::new( | ||
| 5247 | + format!("{}.write({})", sink, codes[0]), | ||
| 5248 | + Some(Nim::Unit), | ||
| 5249 | + )); | ||
| 5250 | + } | ||
| 5251 | + | ||
| 5252 | + // The `Debug` derive expands to `Formatter::debug_struct_fieldN_finish | ||
| 5253 | + // (f, "Point", "x", &self.x, ..)`. Rust renders that as | ||
| 5254 | + // `Point { x: 1, y: 2 }`, which is what is built here rather than | ||
| 5255 | + // calling a std helper we do not have. | ||
| 5256 | + if name.starts_with("debug_struct_field") && name.ends_with("_finish") | ||
| 5257 | + || name == "debug_struct_fields_finish" | ||
| 5258 | + { | ||
| 5259 | + if codes.len() >= 2 && (codes.len() - 2) % 2 == 0 { | ||
| 5260 | + let mut parts = vec![format!("{} & \" {{ \"", codes[1])]; | ||
| 5261 | + let mut first = true; | ||
| 5262 | + for pair in codes[2..].chunks(2) { | ||
| 5263 | + if !first { | ||
| 5264 | + parts.push("\", \"".to_string()); | ||
| 5265 | + } | ||
| 5266 | + first = false; | ||
| 5267 | + parts.push(format!("{} & \": \" & rsDebug({})", pair[0], pair[1])); | ||
| 5268 | + } | ||
| 5269 | + parts.push("\" }\"".to_string()); | ||
| 5270 | + return Ok(self.fmt_value(format!("({})", parts.join(" & ")))); | ||
| 5271 | + } | ||
| 5272 | + } | ||
| 5273 | + // The tuple-struct form: `Point(1, 2)`. | ||
| 5274 | + if name.starts_with("debug_tuple_field") && name.ends_with("_finish") { | ||
| 5275 | + if codes.len() >= 2 { | ||
| 5276 | + let inner: Vec<String> = codes[2..] | ||
| 5277 | + .iter() | ||
| 5278 | + .map(|c| format!("rsDebug({})", c)) | ||
| 5279 | + .collect(); | ||
| 5280 | + return Ok(self.fmt_value(format!( | ||
| 5281 | + "({} & \"(\" & {} & \")\")", | ||
| 5282 | + codes[1], | ||
| 5283 | + inner.join(" & \", \" & ") | ||
| 5284 | + ))); | ||
| 5285 | + } | ||
| 5286 | + } | ||
| 5287 | + | ||
| 5288 | + // `Trait::method(x, ..)` is the same call as `x.method(..)` -- Rust's | ||
| 5289 | + // universal call syntax. Macro expansions lean on it heavily: a | ||
| 5290 | + // generated `Clone::clone(&self.x)` needs nothing in scope. | ||
| 5291 | + if ret.is_none() | ||
| 5292 | + && !self.structs.contains_key(&name) | ||
| 5293 | + && !self.enums.contains_key(&name) | ||
| 5294 | + && p.path.segments.len() >= 2 | ||
| 5295 | + && !c.args.is_empty() | ||
| 5296 | + { | ||
| 5297 | + let mut rest = syn::punctuated::Punctuated::new(); | ||
| 5298 | + for a in c.args.iter().skip(1) { | ||
| 5299 | + rest.push(a.clone()); | ||
| 5300 | + } | ||
| 5301 | + let as_method = syn::ExprMethodCall { | ||
| 5302 | + attrs: Vec::new(), | ||
| 5303 | + receiver: Box::new(c.args[0].clone()), | ||
| 5304 | + dot_token: Default::default(), | ||
| 5305 | + method: syn::Ident::new(&name, proc_macro2::Span::call_site()), | ||
| 5306 | + turbofish: None, | ||
| 5307 | + paren_token: Default::default(), | ||
| 5308 | + args: rest, | ||
| 5309 | + }; | ||
| 5310 | + if let Ok(v) = self.method(&as_method, expect) { | ||
| 5311 | + return Ok(v); | ||
| 5312 | + } | ||
| 5313 | + } | ||
| 5314 | + | ||
| 5228 | if ret.is_none() && !self.structs.contains_key(&name) && !self.enums.contains_key(&name) { | 5315 | if ret.is_none() && !self.structs.contains_key(&name) && !self.enums.contains_key(&name) { |
| 5229 | return Err(format!( | 5316 | return Err(format!( |
| 5230 | "call to unknown function `{name}`; only functions defined in \ | 5317 | "call to unknown function `{name}`; only functions defined in \ |
| @@ -5874,7 +5961,11 @@ impl Lowerer { | |||
| 5874 | (true, false) => format!("stderr.write({s})"), | 5961 | (true, false) => format!("stderr.write({s})"), |
| 5875 | }) | 5962 | }) |
| 5876 | } | 5963 | } |
| 5877 | - "format" => self.format_args(mac), | 5964 | + "format" | "format_args" | "format_args_nl" => { |
| 5965 | + // `format_args!` is what `println!` desugars to once macros are | ||
| 5966 | + // expanded; the trailing newline is already in the string. | ||
| 5967 | + self.format_args(mac) | ||
| 5968 | + } | ||
| 5878 | "write" | "writeln" => { | 5969 | "write" | "writeln" => { |
| 5879 | // `write!(f, "..", ..)` inside a formatting impl: the first | 5970 | // `write!(f, "..", ..)` inside a formatting impl: the first |
| 5880 | // argument is the sink, the rest is an ordinary format call. | 5971 | // argument is the sink, the rest is an ordinary format call. |
modified
src/main.rs +59 -3 | @@ -21,6 +21,34 @@ const THREAD_CFG: &str = "\ | ||
| 21 | 21 | --mm:atomicArc |
| 22 | 22 | "; |
| 23 | 23 | |
| 24 | +/// Run rustc's macro expansion over a file and return the resulting Rust. | |
| 25 | +/// | |
| 26 | +/// `RUSTC_BOOTSTRAP=1` unlocks `-Zunpretty=expanded` on a stable compiler. | |
| 27 | +/// That is what `cargo-expand` does too; it is an unstable interface, so a | |
| 28 | +/// future rustc could change or withdraw it. | |
| 29 | +fn expand_macros(path: &std::path::Path) -> Result<String, String> { | |
| 30 | + let out = std::process::Command::new("rustc") | |
| 31 | + .arg("--edition=2021") | |
| 32 | + .arg("-Zunpretty=expanded") | |
| 33 | + .arg("-A") | |
| 34 | + .arg("warnings") | |
| 35 | + .arg(path) | |
| 36 | + .env("RUSTC_BOOTSTRAP", "1") | |
| 37 | + .output() | |
| 38 | + .map_err(|e| format!("`--expand` needs rustc on PATH: {e}"))?; | |
| 39 | + if !out.status.success() { | |
| 40 | + return Err(format!( | |
| 41 | + "macro expansion failed:\n{}", | |
| 42 | + String::from_utf8_lossy(&out.stderr) | |
| 43 | + .lines() | |
| 44 | + .take(20) | |
| 45 | + .collect::<Vec<_>>() | |
| 46 | + .join("\n") | |
| 47 | + )); | |
| 48 | + } | |
| 49 | + Ok(String::from_utf8_lossy(&out.stdout).into_owned()) | |
| 50 | +} | |
| 51 | + | |
| 24 | 52 | fn main() -> ExitCode { |
| 25 | 53 | match run() { |
| 26 | 54 | Ok(()) => ExitCode::SUCCESS, |
| @@ -40,6 +68,7 @@ fn run() -> Result<(), String> { | ||
| 40 | 68 | let mut output: Option<PathBuf> = None; |
| 41 | 69 | let mut features: Vec<String> = Vec::new(); |
| 42 | 70 | let mut cfgs: Vec<String> = Vec::new(); |
| 71 | + let mut expand = false; | |
| 43 | 72 | |
| 44 | 73 | while let Some(a) = args.next() { |
| 45 | 74 | match a.to_string_lossy().as_ref() { |
| @@ -50,6 +79,7 @@ fn run() -> Result<(), String> { | ||
| 50 | 79 | .into(), |
| 51 | 80 | ); |
| 52 | 81 | } |
| 82 | + "--expand" => expand = true, | |
| 53 | 83 | "--cfg" => { |
| 54 | 84 | let v = args.next().ok_or("`--cfg` needs an argument")?; |
| 55 | 85 | let v = v.to_string_lossy().into_owned(); |
| @@ -63,7 +93,13 @@ fn run() -> Result<(), String> { | ||
| 63 | 93 | } |
| 64 | 94 | } |
| 65 | 95 | "-h" | "--help" => { |
| 66 | - println!("usage: rustnim <input.rs>... [--cfg <name>|feature=<name>]... [-o <output.nim>]"); | |
| 96 | + println!("usage: rustnim [--expand] <input.rs>... [--cfg <name>|feature=<name>]... [-o <output.nim>]"); | |
| 97 | + println!(); | |
| 98 | + println!("--expand runs rustc's macro expansion over the input first, so"); | |
| 99 | + println!("derives and other proc macros are resolved before lowering. A proc"); | |
| 100 | + println!("macro is a compile-time program and cannot be translated, but its"); | |
| 101 | + println!("*output* is ordinary Rust. Needs rustc at transpile time only: what"); | |
| 102 | + println!("rustnim writes is Nim, with no Rust toolchain behind it."); | |
| 67 | 103 | println!(); |
| 68 | 104 | println!("Several inputs are concatenated into one Nim module, in the"); |
| 69 | 105 | println!("order given. That is how a multi-file crate is handled: Nim"); |
| @@ -84,13 +120,33 @@ fn run() -> Result<(), String> { | ||
| 84 | 120 | // inside a single output file, so the items are flattened in argument |
| 85 | 121 | // order. A name collision between two files is a Nim compile error, which |
| 86 | 122 | // is a loud failure rather than a silently shadowed definition. |
| 123 | + // `--expand`: run the macro expansion before lowering. A proc macro is a | |
| 124 | + // program that generates Rust -- it has to be *run*, and rustc is what | |
| 125 | + // runs it. This happens at transpile time, so the Nim that comes out the | |
| 126 | + // other side needs no Rust toolchain. | |
| 127 | + let expanded_src = if expand { | |
| 128 | + if inputs.len() != 1 { | |
| 129 | + return Err( | |
| 130 | + "`--expand` takes a single crate root; for a multi-file crate, expand \ | |
| 131 | + it with `cargo rustc -- -Zunpretty=expanded` and pass the result" | |
| 132 | + .into(), | |
| 133 | + ); | |
| 134 | + } | |
| 135 | + Some(expand_macros(&inputs[0])?) | |
| 136 | + } else { | |
| 137 | + None | |
| 138 | + }; | |
| 139 | + | |
| 87 | 140 | // The first input is the crate root; each later one is a module named by |
| 88 | 141 | // its file stem. That is what keeps `lower::decode` and `mixed::decode` |
| 89 | 142 | // apart once everything is flattened into a single Nim module. |
| 90 | 143 | let mut files = Vec::new(); |
| 91 | 144 | for (i, input) in inputs.iter().enumerate() { |
| 92 | - let src = std::fs::read_to_string(input) | |
| 93 | - .map_err(|e| format!("cannot read {}: {e}", input.display()))?; | |
| 145 | + let src = match &expanded_src { | |
| 146 | + Some(s) => s.clone(), | |
| 147 | + None => std::fs::read_to_string(input) | |
| 148 | + .map_err(|e| format!("cannot read {}: {e}", input.display()))?, | |
| 149 | + }; | |
| 94 | 150 | let parsed: syn::File = syn::parse_file(&src) |
| 95 | 151 | .map_err(|e| format!("{}: parse error: {e}", input.display()))?; |
| 96 | 152 | let module = if i == 0 { |
| @@ -21,6 +21,34 @@ const THREAD_CFG: &str = "\ | |||
| 21 | --mm:atomicArc | 21 | --mm:atomicArc |
| 22 | "; | 22 | "; |
| 23 | 23 | ||
| 24 | +/// Run rustc's macro expansion over a file and return the resulting Rust. | ||
| 25 | +/// | ||
| 26 | +/// `RUSTC_BOOTSTRAP=1` unlocks `-Zunpretty=expanded` on a stable compiler. | ||
| 27 | +/// That is what `cargo-expand` does too; it is an unstable interface, so a | ||
| 28 | +/// future rustc could change or withdraw it. | ||
| 29 | +fn expand_macros(path: &std::path::Path) -> Result<String, String> { | ||
| 30 | + let out = std::process::Command::new("rustc") | ||
| 31 | + .arg("--edition=2021") | ||
| 32 | + .arg("-Zunpretty=expanded") | ||
| 33 | + .arg("-A") | ||
| 34 | + .arg("warnings") | ||
| 35 | + .arg(path) | ||
| 36 | + .env("RUSTC_BOOTSTRAP", "1") | ||
| 37 | + .output() | ||
| 38 | + .map_err(|e| format!("`--expand` needs rustc on PATH: {e}"))?; | ||
| 39 | + if !out.status.success() { | ||
| 40 | + return Err(format!( | ||
| 41 | + "macro expansion failed:\n{}", | ||
| 42 | + String::from_utf8_lossy(&out.stderr) | ||
| 43 | + .lines() | ||
| 44 | + .take(20) | ||
| 45 | + .collect::<Vec<_>>() | ||
| 46 | + .join("\n") | ||
| 47 | + )); | ||
| 48 | + } | ||
| 49 | + Ok(String::from_utf8_lossy(&out.stdout).into_owned()) | ||
| 50 | +} | ||
| 51 | + | ||
| 24 | fn main() -> ExitCode { | 52 | fn main() -> ExitCode { |
| 25 | match run() { | 53 | match run() { |
| 26 | Ok(()) => ExitCode::SUCCESS, | 54 | Ok(()) => ExitCode::SUCCESS, |
| @@ -40,6 +68,7 @@ fn run() -> Result<(), String> { | |||
| 40 | let mut output: Option<PathBuf> = None; | 68 | let mut output: Option<PathBuf> = None; |
| 41 | let mut features: Vec<String> = Vec::new(); | 69 | let mut features: Vec<String> = Vec::new(); |
| 42 | let mut cfgs: Vec<String> = Vec::new(); | 70 | let mut cfgs: Vec<String> = Vec::new(); |
| 71 | + let mut expand = false; | ||
| 43 | 72 | ||
| 44 | while let Some(a) = args.next() { | 73 | while let Some(a) = args.next() { |
| 45 | match a.to_string_lossy().as_ref() { | 74 | match a.to_string_lossy().as_ref() { |
| @@ -50,6 +79,7 @@ fn run() -> Result<(), String> { | |||
| 50 | .into(), | 79 | .into(), |
| 51 | ); | 80 | ); |
| 52 | } | 81 | } |
| 82 | + "--expand" => expand = true, | ||
| 53 | "--cfg" => { | 83 | "--cfg" => { |
| 54 | let v = args.next().ok_or("`--cfg` needs an argument")?; | 84 | let v = args.next().ok_or("`--cfg` needs an argument")?; |
| 55 | let v = v.to_string_lossy().into_owned(); | 85 | let v = v.to_string_lossy().into_owned(); |
| @@ -63,7 +93,13 @@ fn run() -> Result<(), String> { | |||
| 63 | } | 93 | } |
| 64 | } | 94 | } |
| 65 | "-h" | "--help" => { | 95 | "-h" | "--help" => { |
| 66 | - println!("usage: rustnim <input.rs>... [--cfg <name>|feature=<name>]... [-o <output.nim>]"); | 96 | + println!("usage: rustnim [--expand] <input.rs>... [--cfg <name>|feature=<name>]... [-o <output.nim>]"); |
| 97 | + println!(); | ||
| 98 | + println!("--expand runs rustc's macro expansion over the input first, so"); | ||
| 99 | + println!("derives and other proc macros are resolved before lowering. A proc"); | ||
| 100 | + println!("macro is a compile-time program and cannot be translated, but its"); | ||
| 101 | + println!("*output* is ordinary Rust. Needs rustc at transpile time only: what"); | ||
| 102 | + println!("rustnim writes is Nim, with no Rust toolchain behind it."); | ||
| 67 | println!(); | 103 | println!(); |
| 68 | println!("Several inputs are concatenated into one Nim module, in the"); | 104 | println!("Several inputs are concatenated into one Nim module, in the"); |
| 69 | println!("order given. That is how a multi-file crate is handled: Nim"); | 105 | println!("order given. That is how a multi-file crate is handled: Nim"); |
| @@ -84,13 +120,33 @@ fn run() -> Result<(), String> { | |||
| 84 | // inside a single output file, so the items are flattened in argument | 120 | // inside a single output file, so the items are flattened in argument |
| 85 | // order. A name collision between two files is a Nim compile error, which | 121 | // order. A name collision between two files is a Nim compile error, which |
| 86 | // is a loud failure rather than a silently shadowed definition. | 122 | // is a loud failure rather than a silently shadowed definition. |
| 123 | + // `--expand`: run the macro expansion before lowering. A proc macro is a | ||
| 124 | + // program that generates Rust -- it has to be *run*, and rustc is what | ||
| 125 | + // runs it. This happens at transpile time, so the Nim that comes out the | ||
| 126 | + // other side needs no Rust toolchain. | ||
| 127 | + let expanded_src = if expand { | ||
| 128 | + if inputs.len() != 1 { | ||
| 129 | + return Err( | ||
| 130 | + "`--expand` takes a single crate root; for a multi-file crate, expand \ | ||
| 131 | + it with `cargo rustc -- -Zunpretty=expanded` and pass the result" | ||
| 132 | + .into(), | ||
| 133 | + ); | ||
| 134 | + } | ||
| 135 | + Some(expand_macros(&inputs[0])?) | ||
| 136 | + } else { | ||
| 137 | + None | ||
| 138 | + }; | ||
| 139 | + | ||
| 87 | // The first input is the crate root; each later one is a module named by | 140 | // The first input is the crate root; each later one is a module named by |
| 88 | // its file stem. That is what keeps `lower::decode` and `mixed::decode` | 141 | // its file stem. That is what keeps `lower::decode` and `mixed::decode` |
| 89 | // apart once everything is flattened into a single Nim module. | 142 | // apart once everything is flattened into a single Nim module. |
| 90 | let mut files = Vec::new(); | 143 | let mut files = Vec::new(); |
| 91 | for (i, input) in inputs.iter().enumerate() { | 144 | for (i, input) in inputs.iter().enumerate() { |
| 92 | - let src = std::fs::read_to_string(input) | 145 | + let src = match &expanded_src { |
| 93 | - .map_err(|e| format!("cannot read {}: {e}", input.display()))?; | 146 | + Some(s) => s.clone(), |
| 147 | + None => std::fs::read_to_string(input) | ||
| 148 | + .map_err(|e| format!("cannot read {}: {e}", input.display()))?, | ||
| 149 | + }; | ||
| 94 | let parsed: syn::File = syn::parse_file(&src) | 150 | let parsed: syn::File = syn::parse_file(&src) |
| 95 | .map_err(|e| format!("{}: parse error: {e}", input.display()))?; | 151 | .map_err(|e| format!("{}: parse error: {e}", input.display()))?; |
| 96 | let module = if i == 0 { | 152 | let module = if i == 0 { |
added
tests/cases/047-expanded-derives.rs +33 -0 | new file mode 100644 | ||
| @@ -0,0 +1,33 @@ | ||
| 1 | +//@ expand | |
| 2 | +// A proc macro is a compile-time *program*: it cannot be translated, only run. | |
| 3 | +// But its output is ordinary Rust, so rustnim runs the expansion at ingestion | |
| 4 | +// (`--expand`, which is rustc's `-Zunpretty=expanded`) and lowers the result. | |
| 5 | +// The Nim that comes out needs no Rust toolchain behind it. | |
| 6 | +// | |
| 7 | +// This case is transpiled from the *expanded* source, so what is exercised is | |
| 8 | +// what `#[derive(..)]` actually generates: UFCS calls like | |
| 9 | +// `Clone::clone(&self.x)`, the `Formatter::debug_struct_fieldN_finish` helper, | |
| 10 | +// and `println!` desugared to `_print(format_args!(..))`. | |
| 11 | + | |
| 12 | +#[derive(Clone, Copy, Debug, PartialEq)] | |
| 13 | +pub struct Point { | |
| 14 | + pub x: i32, | |
| 15 | + pub y: i32, | |
| 16 | +} | |
| 17 | + | |
| 18 | +#[derive(Clone, Debug, PartialEq)] | |
| 19 | +pub struct Named { | |
| 20 | + pub id: u8, | |
| 21 | + pub label: String, | |
| 22 | +} | |
| 23 | + | |
| 24 | +fn main() { | |
| 25 | + let a = Point { x: 1, y: -2 }; | |
| 26 | + let b = a.clone(); | |
| 27 | + println!("{:?} {} {}", b, a == b, a == Point { x: 9, y: -2 }); | |
| 28 | + | |
| 29 | + let n = Named { id: 7, label: "abc".to_string() }; | |
| 30 | + let m = n.clone(); | |
| 31 | + println!("{:?} {}", m, n == m); | |
| 32 | + println!("{:?}", Named { id: 0, label: "".to_string() }); | |
| 33 | +} | |
| new file mode 100644 | |||
| @@ -0,0 +1,33 @@ | |||
| 1 | +//@ expand | ||
| 2 | +// A proc macro is a compile-time *program*: it cannot be translated, only run. | ||
| 3 | +// But its output is ordinary Rust, so rustnim runs the expansion at ingestion | ||
| 4 | +// (`--expand`, which is rustc's `-Zunpretty=expanded`) and lowers the result. | ||
| 5 | +// The Nim that comes out needs no Rust toolchain behind it. | ||
| 6 | +// | ||
| 7 | +// This case is transpiled from the *expanded* source, so what is exercised is | ||
| 8 | +// what `#[derive(..)]` actually generates: UFCS calls like | ||
| 9 | +// `Clone::clone(&self.x)`, the `Formatter::debug_struct_fieldN_finish` helper, | ||
| 10 | +// and `println!` desugared to `_print(format_args!(..))`. | ||
| 11 | + | ||
| 12 | +#[derive(Clone, Copy, Debug, PartialEq)] | ||
| 13 | +pub struct Point { | ||
| 14 | + pub x: i32, | ||
| 15 | + pub y: i32, | ||
| 16 | +} | ||
| 17 | + | ||
| 18 | +#[derive(Clone, Debug, PartialEq)] | ||
| 19 | +pub struct Named { | ||
| 20 | + pub id: u8, | ||
| 21 | + pub label: String, | ||
| 22 | +} | ||
| 23 | + | ||
| 24 | +fn main() { | ||
| 25 | + let a = Point { x: 1, y: -2 }; | ||
| 26 | + let b = a.clone(); | ||
| 27 | + println!("{:?} {} {}", b, a == b, a == Point { x: 9, y: -2 }); | ||
| 28 | + | ||
| 29 | + let n = Named { id: 7, label: "abc".to_string() }; | ||
| 30 | + let m = n.clone(); | ||
| 31 | + println!("{:?} {}", m, n == m); | ||
| 32 | + println!("{:?}", Named { id: 0, label: "".to_string() }); | ||
| 33 | +} | ||
modified
tests/differential.rs +7 -0 | @@ -134,6 +134,9 @@ struct Directives { | ||
| 134 | 134 | stdin: Option<String>, |
| 135 | 135 | /// `--cfg` flags for rustnim. rustc gets `--cfg feature="x"` to match. |
| 136 | 136 | cfg: Vec<String>, |
| 137 | + /// Transpile the *expanded* source: rustnim runs rustc's macro expansion | |
| 138 | + /// first, so what is lowered is what the derives generated. | |
| 139 | + expand: bool, | |
| 137 | 140 | /// Crates the *oracle* must link. rustnim does not use them — the point |
| 138 | 141 | /// of such a case is that rustnim reproduces the crate's behaviour |
| 139 | 142 | /// without it. |
| @@ -163,6 +166,7 @@ fn directives(src: &str) -> Directives { | ||
| 163 | 166 | "args" => d.args = val.split_whitespace().map(str::to_string).collect(), |
| 164 | 167 | "cfg" => d.cfg.push(val), |
| 165 | 168 | "extern" => d.externs.push(val), |
| 169 | + "expand" => d.expand = true, | |
| 166 | 170 | "stdin" => d.stdin = Some(format!("{val}\n")), |
| 167 | 171 | _ => {} |
| 168 | 172 | } |
| @@ -285,6 +289,9 @@ fn run_case(case: &Path, work: &Path, nim: &Path) -> Outcome { | ||
| 285 | 289 | ); |
| 286 | 290 | let nim_src = dir.join(format!("{mod_name}.nim")); |
| 287 | 291 | let mut cmd = Command::new(RUSTNIM); |
| 292 | + if d.expand { | |
| 293 | + cmd.arg("--expand"); | |
| 294 | + } | |
| 288 | 295 | cmd.arg(&root); |
| 289 | 296 | for e in &extra { |
| 290 | 297 | cmd.arg(e); |
| @@ -134,6 +134,9 @@ struct Directives { | |||
| 134 | stdin: Option<String>, | 134 | stdin: Option<String>, |
| 135 | /// `--cfg` flags for rustnim. rustc gets `--cfg feature="x"` to match. | 135 | /// `--cfg` flags for rustnim. rustc gets `--cfg feature="x"` to match. |
| 136 | cfg: Vec<String>, | 136 | cfg: Vec<String>, |
| 137 | + /// Transpile the *expanded* source: rustnim runs rustc's macro expansion | ||
| 138 | + /// first, so what is lowered is what the derives generated. | ||
| 139 | + expand: bool, | ||
| 137 | /// Crates the *oracle* must link. rustnim does not use them — the point | 140 | /// Crates the *oracle* must link. rustnim does not use them — the point |
| 138 | /// of such a case is that rustnim reproduces the crate's behaviour | 141 | /// of such a case is that rustnim reproduces the crate's behaviour |
| 139 | /// without it. | 142 | /// without it. |
| @@ -163,6 +166,7 @@ fn directives(src: &str) -> Directives { | |||
| 163 | "args" => d.args = val.split_whitespace().map(str::to_string).collect(), | 166 | "args" => d.args = val.split_whitespace().map(str::to_string).collect(), |
| 164 | "cfg" => d.cfg.push(val), | 167 | "cfg" => d.cfg.push(val), |
| 165 | "extern" => d.externs.push(val), | 168 | "extern" => d.externs.push(val), |
| 169 | + "expand" => d.expand = true, | ||
| 166 | "stdin" => d.stdin = Some(format!("{val}\n")), | 170 | "stdin" => d.stdin = Some(format!("{val}\n")), |
| 167 | _ => {} | 171 | _ => {} |
| 168 | } | 172 | } |
| @@ -285,6 +289,9 @@ fn run_case(case: &Path, work: &Path, nim: &Path) -> Outcome { | |||
| 285 | ); | 289 | ); |
| 286 | let nim_src = dir.join(format!("{mod_name}.nim")); | 290 | let nim_src = dir.join(format!("{mod_name}.nim")); |
| 287 | let mut cmd = Command::new(RUSTNIM); | 291 | let mut cmd = Command::new(RUSTNIM); |
| 292 | + if d.expand { | ||
| 293 | + cmd.arg("--expand"); | ||
| 294 | + } | ||
| 288 | cmd.arg(&root); | 295 | cmd.arg(&root); |
| 289 | for e in &extra { | 296 | for e in &extra { |
| 290 | cmd.arg(e); | 297 | cmd.arg(e); |