| Scaffold rustnim: Rust->Nim transpiler, type mapping 1a218c2 nandi 7h ago | 1 | # rustnim — a Rust → Nim transpiler |
| 2 | |
| 3 | ## Status |
| 4 | |
| Add enums, Option/Result, `?`, and the machinery base16ct needs around them b0ccd80 nandithebull 6h ago | 5 | **Transpiling, and measured.** 27 differential cases, 23 behavioural and 4 |
| 6 | rejections, plus 5 unit/integration tests. All green. Run `cargo test`. |
| 7 | |
| 8 | Passing today: functions, `impl` methods, structs, enums (C-like and |
| 9 | data-carrying), `Option`/`Result` with `?`, `let`/`let mut`, the full integer |
| 10 | and float operator set at exact widths, `as` casts, `if`/`while`/`loop`/`for`, |
| 11 | `match` including patterns that bind, `Vec`/slices/arrays, type aliases |
| 12 | (including generic ones), function-typed parameters (`impl Fn(A) -> B`), |
| 13 | multi-file input, `#[cfg]` evaluation, and `println!`/`format!` with `{}`, |
| 14 | `{:?}`, `{:x}`, `{:b}`, positional and inline-named arguments, and |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 15 | zero/space padding. |
| Scaffold rustnim: Rust->Nim transpiler, type mapping 1a218c2 nandi 7h ago | 16 | |
| 17 | ## Why this exists |
| 18 | |
| 19 | We tried [tarekwasfy01/Code-Transpiler](https://github.com/tarekwasfy01/Code-Transpiler), |
| 20 | which advertises `rust` as a source language, on the `base16ct` crate. It emits |
| 21 | empty files and exits 0. The full investigation is in [`findings/`](findings/) |
| 22 | and is published at |
| 23 | https://rickub.com/nandi/code-transpiler-rust-frontend-findings |
| 24 | |
| 25 | The decisive finding, and the reason this is a new project rather than a patch: |
| 26 | its Universal AST cannot represent Rust. `defaultSemanticTypeContract()` in |
| 27 | `internal/backend/semantic_program.go:85` is hardcoded to |
| 28 | |
| 29 | ``` |
| 30 | numeric: binary64, integer_width: unknown, truth: r_compatible, |
| 31 | ownership: unknown, index_base: 1 |
| 32 | ``` |
| 33 | |
| 34 | and `semantic_document.go:1014` *validates* that every contract equals exactly |
| 35 | that, while `typed_operation.go:46` rejects any value model that is not |
| 36 | `tagged_dynamic_binary64`. There is no integer width and no ownership in the |
| 37 | model at all. Code like `base16ct`'s constant-time decoder — |
| 38 | |
| 39 | ```rust |
| 40 | ret += (((0x2fi16 - byte) & (byte - 0x3a)) >> 8) & (byte - 47); |
| 41 | ``` |
| 42 | |
| 43 | — depends on exact 16-bit signed wrapping and arithmetic shift. Lowering that |
| 44 | into a 1-indexed dynamic float64 model produces silently wrong answers. So the |
| 45 | first rule of this project is the one that codebase broke: |
| 46 | |
| 47 | > **Never approximate a semantic you cannot represent. Fail loudly instead.** |
| 48 | |
| 49 | `src/ty.rs` already does this: `i128`/`u128` are rejected with a reason rather |
| 50 | than widened or truncated. |
| 51 | |
| 52 | ## Architecture |
| 53 | |
| 54 | ``` |
| 55 | Rust source ──syn──> syn AST ──lower──> Nim source ──nim c──> binary |
| 56 | ``` |
| 57 | |
| 58 | **The frontend is `syn`, deliberately.** Hand-rolling a Rust grammar is how the |
| 59 | other project went wrong; a correct parser is not the interesting part of this |
| 60 | problem. The interesting part is the lowering, which is where all the work goes. |
| 61 | |
| 62 | Planned modules: |
| 63 | |
| 64 | | file | role | state | |
| 65 | |---|---|---| |
| 66 | | `src/ty.rs` | Rust type → Nim type, exact widths, explicit rejections | written | |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 67 | | `src/lower.rs` | items, statements, expressions → Nim | written | |
| 68 | | `src/fmt.rs` | `println!`/`format!` format-string handling | written | |
| 69 | | `src/prelude.nim` | `Option`/`Result`/panic/`Display`/`Debug` runtime | written | |
| 70 | | `src/main.rs` | CLI: `rustnim <in.rs> -o <out.nim>` | written | |
| 71 | | `tests/differential.rs` | the runner described below | written | |
| 72 | |
| Add enums, Option/Result, `?`, and the machinery base16ct needs around them b0ccd80 nandithebull 6h ago | 73 | ### Enums, `Option` and `Result` |
| 74 | |
| 75 | A C-like enum becomes a plain Nim `enum`, which compares, orders and |
| 76 | `case`-checks the way Rust's does. A data-carrying enum becomes a Nim object |
| 77 | variant — a discriminant enum plus one branch per variant — which is the same |
| 78 | shape the prelude already uses for `Option` and `Result`. Nim requires the |
| 79 | branches of a variant object to have distinct field names, so each payload |
| 80 | field is prefixed with its variant. |
| 81 | |
| 82 | `match` takes one of two forms. Arms that neither bind nor destructure become |
| 83 | a Nim `case`, which is exhaustiveness-checked the way Rust's is. Arms that do |
| 84 | bind become an `if`/`elif` chain with the bindings emitted as `let`s, because |
| 85 | Nim's `case` cannot destructure. The chain always ends in an arm that panics: |
| 86 | Rust proved it unreachable, but Nim cannot see that, and leaving the chain |
| 87 | open would silently fall through instead. |
| 88 | |
| 89 | `Ok`, `Err` and `Some` are emitted with their full type arguments |
| 90 | (`rsOk[T, E](v)`), because Nim cannot infer `E` from an `Ok(v)` alone. That is |
| 91 | why the expected type has to reach a `match` arm as well as a `let`. |
| 92 | |
| 93 | `?` expands to statements — a temporary, a discriminant test, and an early |
| 94 | `return` — which are emitted ahead of the line being built. Rust inserts a |
| 95 | `From::from` on the error there; we accept only the case where the two error |
| 96 | types already agree, rather than assume a conversion is the identity. `?` in a |
| 97 | `while` condition is rejected: the early return would run once before the |
| 98 | loop rather than on each iteration. |
| 99 | |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 100 | ### Type propagation is load-bearing |
| 101 | |
| 102 | Rust infers an unsuffixed integer literal's type from context and falls back |
| 103 | to `i32`; Nim falls back to 64-bit `int`. So `lower.rs` threads an *expected |
| 104 | type* down through every expression — into `let` annotations, call arguments, |
| 105 | `match` patterns, compound assignments and both operands of a binary — and |
| 106 | annotates every binding it emits. Without that, `let x: u8 = 200; x + 100` |
| 107 | means two different things in the two languages. With it, a width the lowering |
| 108 | gets wrong becomes a Nim compile error (a loud failure, reported by the |
| 109 | runner) rather than a wrong answer. |
| Scaffold rustnim: Rust->Nim transpiler, type mapping 1a218c2 nandi 7h ago | 110 | |
| 111 | ## Mapping decisions made so far |
| 112 | |
| 113 | - **Integers**: exact width. `i32`→`int32`, `usize`→`uint`, etc. `i128`/`u128` |
| 114 | rejected. |
| 115 | - **Indexing**: both 0-based. Direct. |
| 116 | - **`&T`** → plain value. **`&mut T`** → `var T` parameter. |
| 117 | - **`&[T]`** → `openArray[T]` in parameter position, `seq[T]` when owned. |
| 118 | `Nim::owned()` performs that conversion. |
| 119 | - **Ownership/borrowck**: ignored. Nim is GC'd; for safe Rust this is sound. |
| 120 | - **`Option`/`Result`** → object variants in the prelude. |
| 121 | - **`match`** → Nim `case` where the arms are simple, `if`/`elif` when arms have |
| 122 | guards or bindings. |
| 123 | - **Rust's expression-orientation** maps well: Nim `if`/`case` are expressions |
| 124 | too, and a proc's trailing expression is its return value. |
| 125 | |
| Settle signed-shr and unsigned-wrap semantics against both compilers 87c9cc8 nandi 7h ago | 126 | ### Settled empirically (Nim 2.2.4 vs rustc 1.98.1, both run) |
| 127 | |
| 128 | 1. **Nim's `shr` on a signed integer is arithmetic**, matching Rust. |
| 129 | `int16(-256) shr 8` = `-1` in Nim; `(-256i16) >> 8` = `-1` in Rust. |
| 130 | `base16ct`'s decoder depends on this, so it maps directly with no helper. |
| 131 | 2. **Nim's fixed-width unsigned arithmetic wraps silently**, matching Rust's |
| 132 | `wrapping_*`. `uint8(200) + 100` = `44` in Nim; `200u8.wrapping_add(100)` |
| 133 | = `44` in Rust. So `wrapping_add` on an unsigned type is just `+`. |
| 134 | |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 135 | 3. **We model rustc's debug profile.** Rust debug builds panic on signed |
| 136 | integer overflow; Nim's default build raises `OverflowDefect` on it. Those |
| 137 | are the matching pair, so the runner invokes `rustc` without `-O` and `nim |
| 138 | c` with its defaults, and `tests/cases/016` pins the behaviour. A Rust |
| 139 | panic exits 101 where a Nim Defect exits 1, so every generated module ends |
| 140 | with a handler that maps one to the other — otherwise the runner's |
| 141 | exit-status comparison would be vacuous. `wrapping_*` is therefore an |
| 142 | explicit operation on both sides: unsigned maps to the bare operator (item |
| 143 | 2), signed is routed through the unsigned view of the same width. |
| 144 | 4. **`char` round-trips.** Rust `char` → Nim `Rune`, confirmed for ASCII and |
| 145 | non-ASCII scalars in both `{}` and `{:?}`, and across `as u32` |
| 146 | (`tests/cases/014`). |
| 147 | |
| Settle signed-shr and unsigned-wrap semantics against both compilers 87c9cc8 nandi 7h ago | 148 | ### Still open |
| 149 | |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 150 | 5. `checked_*` and `saturating_*` are not mapped yet; they are currently |
| 151 | rejected as unsupported methods rather than approximated. |
| Add enums, Option/Result, `?`, and the machinery base16ct needs around them b0ccd80 nandithebull 6h ago | 152 | 6. Generics (type and const parameters), traits and trait impls, closures and |
| 153 | iterator adaptors are rejected with a reason. Lifetime parameters are *not* |
| 154 | a rejection: they carry no runtime meaning and Nim is GC'd, so |
| 155 | `fn encode<'a>(..)` lowers fine. |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 156 | 7. Float formatting matches Rust for ordinary values and for `inf`/`NaN`, but |
| 157 | the exponent-form thresholds have only been checked at `1e21`. |
| Add enums, Option/Result, `?`, and the machinery base16ct needs around them b0ccd80 nandithebull 6h ago | 158 | 8. Flattening several files into one module can collide a crate's own |
| 159 | `type Result<T>` with the builtin `Result<T, E>`. Rust kept them apart by |
| 160 | module; we tell them apart by arity. That is a real difference from Rust's |
| 161 | resolution and would need proper module scoping to fix. |
| Scaffold rustnim: Rust->Nim transpiler, type mapping 1a218c2 nandi 7h ago | 162 | |
| 163 | ## Testing: differential, not golden |
| 164 | |
| 165 | The bar is **behavioural equivalence with rustc**, not that the output looks |
| 166 | plausible. For each case in `tests/cases/`: |
| 167 | |
| 168 | ``` |
| 169 | rustc case.rs && ./case > expected |
| 170 | rustnim case.rs -o case.nim && nim c -r case.nim > actual |
| 171 | diff expected actual |
| 172 | ``` |
| 173 | |
| 174 | A case only counts as passing when both binaries build *and* produce identical |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 175 | stdout *and* exit with the same status. |
| 176 | |
| 177 | `tests/differential.rs` implements this, and checks each stage separately so a |
| 178 | failure says where it went wrong: `rustnim`, `rustc`, `nim`, or `diff`. Three |
| 179 | guards exist specifically because of how the other transpiler failed: |
| 180 | |
| 181 | - `rustnim` exiting 0 while writing **no output file** is a failure. |
| 182 | - `rustnim` exiting 0 while writing an **empty output file** is a failure. |
| 183 | - An **empty corpus** is a failure, so the runner cannot pass by finding |
| 184 | nothing to do. |
| 185 | |
| 186 | All three have been verified by deliberately breaking the transpiler and |
| 187 | confirming the runner goes red. |
| 188 | |
| 189 | Cases carry directives in leading `//@` comments: |
| 190 | |
| 191 | | directive | meaning | |
| 192 | |---|---| |
| 193 | | `//@ reject: <substring>` | `rustnim` must *fail*, with this in its message | |
| 194 | | `//@ skip: <reason>` | not run; reported as skipped | |
| 195 | | `//@ args: <argv>` | passed to both binaries | |
| 196 | | `//@ stdin: <line>` | fed to both binaries | |
| 197 | |
| 198 | `reject` cases are how the "fail loudly" rule is tested rather than merely |
| 199 | stated: `900`–`904` pin the rejections of `i128`, an unmapped standard-library |
| 200 | method, a float→int cast, an unimplemented format spec, and a closure. |
| 201 | |
| 202 | Run one case with `RUSTNIM_CASE=005 cargo test --test differential -- |
| 203 | --nocapture`. Nim is found at `.nim-toolchain/bin/nim` in the repository root |
| 204 | or any parent, or via `RUSTNIM_NIM`. |
| Scaffold rustnim: Rust->Nim transpiler, type mapping 1a218c2 nandi 7h ago | 205 | |
| 206 | ## Toolchain |
| 207 | |
| 208 | - `rustc` / `cargo` 1.98.1 — system. |
| 209 | - Nim 2.2.4 — vendored at `.nim-toolchain/` (gitignored; downloaded from |
| 210 | nim-lang.org, not installed system-wide). Binary: `.nim-toolchain/bin/nim`. |
| 211 | |
| 212 | ## Milestone 1 |
| 213 | |
| 214 | Transpile `base16ct` 1.0.0 — the crate the other transpiler failed on — and |
| Add enums, Option/Result, `?`, and the machinery base16ct needs around them b0ccd80 nandithebull 6h ago | 215 | have its decoder produce byte-identical output to the Rust original. |
| 216 | |
| 217 | **Not reached.** What is reached, and measured, is `tests/cases/022`: the |
| 218 | `Error` enum, `decoded_len` and `decode_nibble` **verbatim from base16ct |
| 219 | 1.0.0**, decoding lower, upper and mixed hex and both error cases, with output |
| 220 | byte-identical to rustc's. That covers the constant-time nibble arithmetic — |
| 221 | the i16 wrapping and arithmetic shift whose exact semantics the other |
| 222 | transpiler's float64 universal AST cannot represent at all. The loop around it |
| 223 | is rewritten with indexing, and the case says so. |
| 224 | |
| 225 | Running the real crate now gives these diagnostics, which are the todo list: |
| 226 | |
| 227 | | file | blocker | |
| 228 | |---|---| |
| 229 | | `error.rs` | `impl fmt::Display for Error`, `impl core::error::Error`, `impl From<Error> for fmt::Error` — trait impls | |
| 230 | | `lib.rs` | `decode_inner`: `dst.get_mut(..n)` (a mutable subslice view), `chunks_exact(2)`, `zip`, `iter_mut`, and `*dst = ..` | |
| 231 | | `lower.rs`, `upper.rs`, `mixed.rs` | the same, plus `encode`'s `chunks_exact_mut` | |
| 232 | | `display.rs` | `impl fmt::UpperHex for HexDisplay` — trait impls again | |
| 233 | |
| 234 | So the remaining work is two features, not a long tail: **trait impls**, and |
| 235 | **iterator adaptors over slices** together with the mutable slice views they |
| 236 | borrow from. `mod`/multi-file and `#[cfg]` are done; pass the crate's files |
| 237 | together and add `--cfg feature=alloc` for the `alloc` half. |