| Scaffold rustnim: Rust->Nim transpiler, type mapping 1a218c2 nandi 8h ago | 1 | # rustnim — a Rust → Nim transpiler |
| 2 | |
| 3 | ## Status |
| 4 | |
| Add trait impls and slice iterators; base16ct's decoder now goes through ae9f986 nandithebull 6h ago | 5 | **Milestone 1's decoder goal is reached.** 31 differential cases, 27 |
| 6 | behavioural and 4 rejections, plus 5 unit/integration tests. All green. Run |
| 7 | `cargo test`. |
| 8 | |
| 9 | Passing today: functions, `impl` methods, trait impls (formatting traits and |
| 10 | `From`), structs, enums (C-like and data-carrying), `Option`/`Result` with |
| 11 | `?`, slice iterators (`iter`/`iter_mut`/`enumerate`/`zip`/`chunks_exact`/ |
| 12 | `chunks_exact_mut`/`windows`), borrowed slices as values and return types, |
| 13 | `let`/`let mut`, the full integer |
| Add enums, Option/Result, `?`, and the machinery base16ct needs around them b0ccd80 nandithebull 7h ago | 14 | and float operator set at exact widths, `as` casts, `if`/`while`/`loop`/`for`, |
| 15 | `match` including patterns that bind, `Vec`/slices/arrays, type aliases |
| 16 | (including generic ones), function-typed parameters (`impl Fn(A) -> B`), |
| 17 | multi-file input, `#[cfg]` evaluation, and `println!`/`format!` with `{}`, |
| 18 | `{:?}`, `{:x}`, `{:b}`, positional and inline-named arguments, and |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 19 | zero/space padding. |
| Scaffold rustnim: Rust->Nim transpiler, type mapping 1a218c2 nandi 8h ago | 20 | |
| 21 | ## Why this exists |
| 22 | |
| 23 | We tried [tarekwasfy01/Code-Transpiler](https://github.com/tarekwasfy01/Code-Transpiler), |
| 24 | which advertises `rust` as a source language, on the `base16ct` crate. It emits |
| 25 | empty files and exits 0. The full investigation is in [`findings/`](findings/) |
| 26 | and is published at |
| 27 | https://rickub.com/nandi/code-transpiler-rust-frontend-findings |
| 28 | |
| 29 | The decisive finding, and the reason this is a new project rather than a patch: |
| 30 | its Universal AST cannot represent Rust. `defaultSemanticTypeContract()` in |
| 31 | `internal/backend/semantic_program.go:85` is hardcoded to |
| 32 | |
| 33 | ``` |
| 34 | numeric: binary64, integer_width: unknown, truth: r_compatible, |
| 35 | ownership: unknown, index_base: 1 |
| 36 | ``` |
| 37 | |
| 38 | and `semantic_document.go:1014` *validates* that every contract equals exactly |
| 39 | that, while `typed_operation.go:46` rejects any value model that is not |
| 40 | `tagged_dynamic_binary64`. There is no integer width and no ownership in the |
| 41 | model at all. Code like `base16ct`'s constant-time decoder — |
| 42 | |
| 43 | ```rust |
| 44 | ret += (((0x2fi16 - byte) & (byte - 0x3a)) >> 8) & (byte - 47); |
| 45 | ``` |
| 46 | |
| 47 | — depends on exact 16-bit signed wrapping and arithmetic shift. Lowering that |
| 48 | into a 1-indexed dynamic float64 model produces silently wrong answers. So the |
| 49 | first rule of this project is the one that codebase broke: |
| 50 | |
| 51 | > **Never approximate a semantic you cannot represent. Fail loudly instead.** |
| 52 | |
| 53 | `src/ty.rs` already does this: `i128`/`u128` are rejected with a reason rather |
| 54 | than widened or truncated. |
| 55 | |
| 56 | ## Architecture |
| 57 | |
| 58 | ``` |
| 59 | Rust source ──syn──> syn AST ──lower──> Nim source ──nim c──> binary |
| 60 | ``` |
| 61 | |
| 62 | **The frontend is `syn`, deliberately.** Hand-rolling a Rust grammar is how the |
| 63 | other project went wrong; a correct parser is not the interesting part of this |
| 64 | problem. The interesting part is the lowering, which is where all the work goes. |
| 65 | |
| 66 | Planned modules: |
| 67 | |
| 68 | | file | role | state | |
| 69 | |---|---|---| |
| 70 | | `src/ty.rs` | Rust type → Nim type, exact widths, explicit rejections | written | |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 71 | | `src/lower.rs` | items, statements, expressions → Nim | written | |
| 72 | | `src/fmt.rs` | `println!`/`format!` format-string handling | written | |
| 73 | | `src/prelude.nim` | `Option`/`Result`/panic/`Display`/`Debug` runtime | written | |
| 74 | | `src/main.rs` | CLI: `rustnim <in.rs> -o <out.nim>` | written | |
| 75 | | `tests/differential.rs` | the runner described below | written | |
| 76 | |
| Add enums, Option/Result, `?`, and the machinery base16ct needs around them b0ccd80 nandithebull 7h ago | 77 | ### Enums, `Option` and `Result` |
| 78 | |
| 79 | A C-like enum becomes a plain Nim `enum`, which compares, orders and |
| 80 | `case`-checks the way Rust's does. A data-carrying enum becomes a Nim object |
| 81 | variant — a discriminant enum plus one branch per variant — which is the same |
| 82 | shape the prelude already uses for `Option` and `Result`. Nim requires the |
| 83 | branches of a variant object to have distinct field names, so each payload |
| 84 | field is prefixed with its variant. |
| 85 | |
| 86 | `match` takes one of two forms. Arms that neither bind nor destructure become |
| 87 | a Nim `case`, which is exhaustiveness-checked the way Rust's is. Arms that do |
| 88 | bind become an `if`/`elif` chain with the bindings emitted as `let`s, because |
| 89 | Nim's `case` cannot destructure. The chain always ends in an arm that panics: |
| 90 | Rust proved it unreachable, but Nim cannot see that, and leaving the chain |
| 91 | open would silently fall through instead. |
| 92 | |
| 93 | `Ok`, `Err` and `Some` are emitted with their full type arguments |
| 94 | (`rsOk[T, E](v)`), because Nim cannot infer `E` from an `Ok(v)` alone. That is |
| 95 | why the expected type has to reach a `match` arm as well as a `let`. |
| 96 | |
| 97 | `?` expands to statements — a temporary, a discriminant test, and an early |
| 98 | `return` — which are emitted ahead of the line being built. Rust inserts a |
| 99 | `From::from` on the error there; we accept only the case where the two error |
| 100 | types already agree, rather than assume a conversion is the identity. `?` in a |
| 101 | `while` condition is rejected: the early return would run once before the |
| 102 | loop rather than on each iteration. |
| 103 | |
| Add trait impls and slice iterators; base16ct's decoder now goes through ae9f986 nandithebull 6h ago | 104 | ### Trait impls |
| 105 | |
| 106 | A `Display` impl becomes `proc rsDisplay(self: T): string`. Rust's `Formatter` |
| 107 | is a sink and the observable result of `{}` is exactly the bytes written into |
| 108 | it, so every write through the formatter produces that string and the existing |
| 109 | `match`/trailing-expression machinery assembles it. A `fmt` body that does |
| 110 | anything else with the formatter — padding, precision, `debug_struct` — is |
| 111 | rejected, because those change the output and this model does not carry them. |
| 112 | `Debug`, `LowerHex`, `UpperHex`, `Binary` and `Octal` work the same way. |
| 113 | |
| 114 | `impl From<A> for B` becomes a conversion proc that `.into()` resolves |
| 115 | through. A marker trait with no items generates nothing: we do not model trait |
| 116 | resolution anywhere, so there is nothing for it to affect; a use that actually |
| 117 | needed the trait (a `dyn`, a bound) is rejected where it appears. Any other |
| 118 | trait impl is rejected. |
| 119 | |
| 120 | Methods are keyed by `(receiver type, name)`, not by name alone — two types |
| 121 | may define the same method, and Nim tells them apart by overload resolution on |
| 122 | the first parameter. |
| 123 | |
| 124 | `fmt::Error` is *not* the same type as a crate's own `Error`. Collapsing a |
| 125 | qualified path to its last segment merged them, which was a real soundness |
| 126 | bug; `core::fmt`'s types are now recognised by their qualified name. |
| 127 | |
| 128 | ### Slice iterators are resolved to one index loop |
| 129 | |
| 130 | Rust's slice iterators are lazy and compose. Nim's `for` is over one sequence, |
| 131 | so a chain of adaptors is resolved into a small IR and emitted as a single |
| 132 | index loop in which **each binding is an lvalue into the original container**. |
| 133 | That is what makes `*d = v` through `iter_mut()` write back to the caller's |
| 134 | slice instead of to a copy, and what lets `chunks_exact(2)` hand out a window |
| 135 | that indexes straight into the source with an offset. |
| 136 | |
| 137 | Only adaptors with an exact index-loop equivalent are accepted. `map`, |
| 138 | `filter` and `take_while` are rejected rather than partially honoured: |
| 139 | silently dropping an adaptor would change which elements the loop visits. |
| 140 | |
| 141 | `zip` stops at the shorter side, as Rust's does — that is a test, not an |
| 142 | assumption (`tests/cases/023`). |
| 143 | |
| 144 | ### Borrowed slices are views, not copies |
| 145 | |
| 146 | `&[T]` is a borrow. Nim's experimental view types model exactly that, |
| 147 | including returning one from a proc: writing through the returned view is |
| 148 | visible in the original buffer. That was probed against Nim 2.2.4 before being |
| 149 | relied on, because copying into a `seq` would print the right bytes while |
| 150 | silently changing aliasing. |
| 151 | |
| 152 | `s.get(a..b)` is the one place this leaks. It is an `Option<&[T]>`, and Nim |
| 153 | cannot put a view inside an object, so there is no value to hand back. Instead |
| 154 | the view and its validity condition travel together through `ok_or` until a |
| 155 | `?` or `unwrap` resolves them into a bounds check plus a binding. Keeping such |
| 156 | an `Option` in a variable is rejected with a message saying so. |
| 157 | |
| 158 | ### Declaration order |
| 159 | |
| 160 | Rust has no declaration-before-use rule and Nim does, so every proc is |
| 161 | forward-declared between the type definitions and the bodies. Reordering the |
| 162 | input instead would not handle mutual recursion. |
| 163 | |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 164 | ### Type propagation is load-bearing |
| 165 | |
| 166 | Rust infers an unsuffixed integer literal's type from context and falls back |
| 167 | to `i32`; Nim falls back to 64-bit `int`. So `lower.rs` threads an *expected |
| 168 | type* down through every expression — into `let` annotations, call arguments, |
| 169 | `match` patterns, compound assignments and both operands of a binary — and |
| 170 | annotates every binding it emits. Without that, `let x: u8 = 200; x + 100` |
| 171 | means two different things in the two languages. With it, a width the lowering |
| 172 | gets wrong becomes a Nim compile error (a loud failure, reported by the |
| 173 | runner) rather than a wrong answer. |
| Scaffold rustnim: Rust->Nim transpiler, type mapping 1a218c2 nandi 8h ago | 174 | |
| 175 | ## Mapping decisions made so far |
| 176 | |
| 177 | - **Integers**: exact width. `i32`→`int32`, `usize`→`uint`, etc. `i128`/`u128` |
| 178 | rejected. |
| 179 | - **Indexing**: both 0-based. Direct. |
| 180 | - **`&T`** → plain value. **`&mut T`** → `var T` parameter. |
| 181 | - **`&[T]`** → `openArray[T]` in parameter position, `seq[T]` when owned. |
| 182 | `Nim::owned()` performs that conversion. |
| 183 | - **Ownership/borrowck**: ignored. Nim is GC'd; for safe Rust this is sound. |
| 184 | - **`Option`/`Result`** → object variants in the prelude. |
| 185 | - **`match`** → Nim `case` where the arms are simple, `if`/`elif` when arms have |
| 186 | guards or bindings. |
| 187 | - **Rust's expression-orientation** maps well: Nim `if`/`case` are expressions |
| 188 | too, and a proc's trailing expression is its return value. |
| 189 | |
| Settle signed-shr and unsigned-wrap semantics against both compilers 87c9cc8 nandi 8h ago | 190 | ### Settled empirically (Nim 2.2.4 vs rustc 1.98.1, both run) |
| 191 | |
| 192 | 1. **Nim's `shr` on a signed integer is arithmetic**, matching Rust. |
| 193 | `int16(-256) shr 8` = `-1` in Nim; `(-256i16) >> 8` = `-1` in Rust. |
| 194 | `base16ct`'s decoder depends on this, so it maps directly with no helper. |
| 195 | 2. **Nim's fixed-width unsigned arithmetic wraps silently**, matching Rust's |
| 196 | `wrapping_*`. `uint8(200) + 100` = `44` in Nim; `200u8.wrapping_add(100)` |
| 197 | = `44` in Rust. So `wrapping_add` on an unsigned type is just `+`. |
| 198 | |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 199 | 3. **We model rustc's debug profile.** Rust debug builds panic on signed |
| 200 | integer overflow; Nim's default build raises `OverflowDefect` on it. Those |
| 201 | are the matching pair, so the runner invokes `rustc` without `-O` and `nim |
| 202 | c` with its defaults, and `tests/cases/016` pins the behaviour. A Rust |
| 203 | panic exits 101 where a Nim Defect exits 1, so every generated module ends |
| 204 | with a handler that maps one to the other — otherwise the runner's |
| 205 | exit-status comparison would be vacuous. `wrapping_*` is therefore an |
| 206 | explicit operation on both sides: unsigned maps to the bare operator (item |
| 207 | 2), signed is routed through the unsigned view of the same width. |
| 208 | 4. **`char` round-trips.** Rust `char` → Nim `Rune`, confirmed for ASCII and |
| 209 | non-ASCII scalars in both `{}` and `{:?}`, and across `as u32` |
| 210 | (`tests/cases/014`). |
| 211 | |
| Settle signed-shr and unsigned-wrap semantics against both compilers 87c9cc8 nandi 8h ago | 212 | ### Still open |
| 213 | |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 214 | 5. `checked_*` and `saturating_*` are not mapped yet; they are currently |
| 215 | rejected as unsupported methods rather than approximated. |
| Add trait impls and slice iterators; base16ct's decoder now goes through ae9f986 nandithebull 6h ago | 216 | 6. Generics (type and const parameters), closures, `unsafe`, and trait impls |
| 217 | other than the formatting traits and `From` are rejected with a reason. |
| 218 | Lifetime parameters are *not* a rejection: they carry no runtime meaning |
| 219 | and Nim is GC'd, so `fn encode<'a>(..)` lowers fine. |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 220 | 7. Float formatting matches Rust for ordinary values and for `inf`/`NaN`, but |
| 221 | the exponent-form thresholds have only been checked at `1e21`. |
| Add enums, Option/Result, `?`, and the machinery base16ct needs around them b0ccd80 nandithebull 7h ago | 222 | 8. Flattening several files into one module can collide a crate's own |
| 223 | `type Result<T>` with the builtin `Result<T, E>`. Rust kept them apart by |
| 224 | module; we tell them apart by arity. That is a real difference from Rust's |
| 225 | resolution and would need proper module scoping to fix. |
| Scaffold rustnim: Rust->Nim transpiler, type mapping 1a218c2 nandi 8h ago | 226 | |
| 227 | ## Testing: differential, not golden |
| 228 | |
| 229 | The bar is **behavioural equivalence with rustc**, not that the output looks |
| 230 | plausible. For each case in `tests/cases/`: |
| 231 | |
| 232 | ``` |
| 233 | rustc case.rs && ./case > expected |
| 234 | rustnim case.rs -o case.nim && nim c -r case.nim > actual |
| 235 | diff expected actual |
| 236 | ``` |
| 237 | |
| 238 | A case only counts as passing when both binaries build *and* produce identical |
| Add the differential test runner, and a lowering to measure with it 8ac32af nandithebull 7h ago | 239 | stdout *and* exit with the same status. |
| 240 | |
| 241 | `tests/differential.rs` implements this, and checks each stage separately so a |
| 242 | failure says where it went wrong: `rustnim`, `rustc`, `nim`, or `diff`. Three |
| 243 | guards exist specifically because of how the other transpiler failed: |
| 244 | |
| 245 | - `rustnim` exiting 0 while writing **no output file** is a failure. |
| 246 | - `rustnim` exiting 0 while writing an **empty output file** is a failure. |
| 247 | - An **empty corpus** is a failure, so the runner cannot pass by finding |
| 248 | nothing to do. |
| 249 | |
| 250 | All three have been verified by deliberately breaking the transpiler and |
| 251 | confirming the runner goes red. |
| 252 | |
| 253 | Cases carry directives in leading `//@` comments: |
| 254 | |
| 255 | | directive | meaning | |
| 256 | |---|---| |
| 257 | | `//@ reject: <substring>` | `rustnim` must *fail*, with this in its message | |
| 258 | | `//@ skip: <reason>` | not run; reported as skipped | |
| 259 | | `//@ args: <argv>` | passed to both binaries | |
| 260 | | `//@ stdin: <line>` | fed to both binaries | |
| 261 | |
| 262 | `reject` cases are how the "fail loudly" rule is tested rather than merely |
| 263 | stated: `900`–`904` pin the rejections of `i128`, an unmapped standard-library |
| 264 | method, a float→int cast, an unimplemented format spec, and a closure. |
| 265 | |
| 266 | Run one case with `RUSTNIM_CASE=005 cargo test --test differential -- |
| 267 | --nocapture`. Nim is found at `.nim-toolchain/bin/nim` in the repository root |
| 268 | or any parent, or via `RUSTNIM_NIM`. |
| Scaffold rustnim: Rust->Nim transpiler, type mapping 1a218c2 nandi 8h ago | 269 | |
| 270 | ## Toolchain |
| 271 | |
| 272 | - `rustc` / `cargo` 1.98.1 — system. |
| 273 | - Nim 2.2.4 — vendored at `.nim-toolchain/` (gitignored; downloaded from |
| 274 | nim-lang.org, not installed system-wide). Binary: `.nim-toolchain/bin/nim`. |
| 275 | |
| 276 | ## Milestone 1 |
| 277 | |
| 278 | Transpile `base16ct` 1.0.0 — the crate the other transpiler failed on — and |
| Add enums, Option/Result, `?`, and the machinery base16ct needs around them b0ccd80 nandithebull 7h ago | 279 | have its decoder produce byte-identical output to the Rust original. |
| 280 | |
| Add trait impls and slice iterators; base16ct's decoder now goes through ae9f986 nandithebull 6h ago | 281 | **The decoder is reached.** `tests/cases/026-base16ct-crate/` transpiles |
| 282 | base16ct's `error.rs` and `mixed.rs` **byte-for-byte as published on |
| 283 | crates.io** — verified with `cmp`, not by eye — together with `lib.rs`'s |
| 284 | `decoded_len`, `encoded_len` and `decode_inner` verbatim, and its output is |
| 285 | byte-identical to rustc's: |
| Add enums, Option/Result, `?`, and the machinery base16ct needs around them b0ccd80 nandithebull 7h ago | 286 | |
| Add trait impls and slice iterators; base16ct's decoder now goes through ae9f986 nandithebull 6h ago | 287 | ``` |
| 288 | mixed-l ok abcd1234 len=4 lower, upper and mixed hex all decode |
| 289 | mixed-u ok abcd1234 len=4 |
| 290 | mixed-m ok abcd1234 len=4 |
| 291 | edge ok 00ff7f80 len=4 |
| 292 | oddlen err InvalidLength / invalid Base16 length <- Debug and Display |
| 293 | bad err InvalidEncoding / invalid Base16 encoding |
| 294 | empty ok len=0 |
| 295 | short-dst err InvalidLength / invalid Base16 length |
| 296 | ``` |
| Add enums, Option/Result, `?`, and the machinery base16ct needs around them b0ccd80 nandithebull 7h ago | 297 | |
| Add trait impls and slice iterators; base16ct's decoder now goes through ae9f986 nandithebull 6h ago | 298 | `decode_inner` goes through as written: `dst.get_mut(..decoded_len(src)?)`, |
| 299 | `src.chunks_exact(2).zip(dst.iter_mut())`, `*dst = byte as u8`, and the |
| 300 | returned `&'a [u8]` view into the caller's buffer. The `Display` line in that |
| 301 | output comes from the crate's own `impl fmt::Display for Error`. |
| 302 | |
| 303 | ### Still to do for the whole crate |
| 304 | |
| 305 | - `lower.rs` / `upper.rs`: `encode` itself lowers, but the same file defines |
| 306 | `encode_str`, whose body is a **closure** over an **`unsafe`** block. Both |
| 307 | are unimplemented, and a file is all-or-nothing, so neither module is in the |
| 308 | case yet. |
| 309 | - `display.rs`: `HexDisplay<'a>(pub &'a [u8])` is a tuple struct holding a |
| 310 | borrowed slice. A struct *field* of view type is what Nim's view types do |
| 311 | not allow, which is the same wall as `Option<&[T]>`. |
| 312 | - The `alloc` half (`decode_vec`, `encode_string`) needs `--cfg feature=alloc` |
| 313 | and then `String::from_utf8_unchecked`, i.e. `unsafe` again. |