rustnim — a Rust → Nim transpiler

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237

# rustnim — a Rust → Nim transpiler

## Status

**Transpiling, and measured.** 27 differential cases, 23 behavioural and 4
rejections, plus 5 unit/integration tests. All green. Run `cargo test`.

Passing today: functions, `impl` methods, structs, enums (C-like and
data-carrying), `Option`/`Result` with `?`, `let`/`let mut`, the full integer
and float operator set at exact widths, `as` casts, `if`/`while`/`loop`/`for`,
`match` including patterns that bind, `Vec`/slices/arrays, type aliases
(including generic ones), function-typed parameters (`impl Fn(A) -> B`),
multi-file input, `#[cfg]` evaluation, and `println!`/`format!` with `{}`,
`{:?}`, `{:x}`, `{:b}`, positional and inline-named arguments, and
zero/space padding.

## Why this exists

We tried [tarekwasfy01/Code-Transpiler](https://github.com/tarekwasfy01/Code-Transpiler),
which advertises `rust` as a source language, on the `base16ct` crate. It emits
empty files and exits 0. The full investigation is in [`findings/`](findings/)
and is published at
https://rickub.com/nandi/code-transpiler-rust-frontend-findings

The decisive finding, and the reason this is a new project rather than a patch:
its Universal AST cannot represent Rust. `defaultSemanticTypeContract()` in
`internal/backend/semantic_program.go:85` is hardcoded to

```
numeric: binary64, integer_width: unknown, truth: r_compatible,
ownership: unknown, index_base: 1
```

and `semantic_document.go:1014` *validates* that every contract equals exactly
that, while `typed_operation.go:46` rejects any value model that is not
`tagged_dynamic_binary64`. There is no integer width and no ownership in the
model at all. Code like `base16ct`'s constant-time decoder —

```rust
ret += (((0x2fi16 - byte) & (byte - 0x3a)) >> 8) & (byte - 47);
```

— depends on exact 16-bit signed wrapping and arithmetic shift. Lowering that
into a 1-indexed dynamic float64 model produces silently wrong answers. So the
first rule of this project is the one that codebase broke:

> **Never approximate a semantic you cannot represent. Fail loudly instead.**

`src/ty.rs` already does this: `i128`/`u128` are rejected with a reason rather
than widened or truncated.

## Architecture

```
Rust source ──syn──> syn AST ──lower──> Nim source ──nim c──> binary
```

**The frontend is `syn`, deliberately.** Hand-rolling a Rust grammar is how the
other project went wrong; a correct parser is not the interesting part of this
problem. The interesting part is the lowering, which is where all the work goes.

Planned modules:

| file | role | state |
|---|---|---|
| `src/ty.rs` | Rust type → Nim type, exact widths, explicit rejections | written |
| `src/lower.rs` | items, statements, expressions → Nim | written |
| `src/fmt.rs` | `println!`/`format!` format-string handling | written |
| `src/prelude.nim` | `Option`/`Result`/panic/`Display`/`Debug` runtime | written |
| `src/main.rs` | CLI: `rustnim <in.rs> -o <out.nim>` | written |
| `tests/differential.rs` | the runner described below | written |

### Enums, `Option` and `Result`

A C-like enum becomes a plain Nim `enum`, which compares, orders and
`case`-checks the way Rust's does. A data-carrying enum becomes a Nim object
variant — a discriminant enum plus one branch per variant — which is the same
shape the prelude already uses for `Option` and `Result`. Nim requires the
branches of a variant object to have distinct field names, so each payload
field is prefixed with its variant.

`match` takes one of two forms. Arms that neither bind nor destructure become
a Nim `case`, which is exhaustiveness-checked the way Rust's is. Arms that do
bind become an `if`/`elif` chain with the bindings emitted as `let`s, because
Nim's `case` cannot destructure. The chain always ends in an arm that panics:
Rust proved it unreachable, but Nim cannot see that, and leaving the chain
open would silently fall through instead.

`Ok`, `Err` and `Some` are emitted with their full type arguments
(`rsOk[T, E](v)`), because Nim cannot infer `E` from an `Ok(v)` alone. That is
why the expected type has to reach a `match` arm as well as a `let`.

`?` expands to statements — a temporary, a discriminant test, and an early
`return` — which are emitted ahead of the line being built. Rust inserts a
`From::from` on the error there; we accept only the case where the two error
types already agree, rather than assume a conversion is the identity. `?` in a
`while` condition is rejected: the early return would run once before the
loop rather than on each iteration.

### Type propagation is load-bearing

Rust infers an unsuffixed integer literal's type from context and falls back
to `i32`; Nim falls back to 64-bit `int`. So `lower.rs` threads an *expected
type* down through every expression — into `let` annotations, call arguments,
`match` patterns, compound assignments and both operands of a binary — and
annotates every binding it emits. Without that, `let x: u8 = 200; x + 100`
means two different things in the two languages. With it, a width the lowering
gets wrong becomes a Nim compile error (a loud failure, reported by the
runner) rather than a wrong answer.

## Mapping decisions made so far

- **Integers**: exact width. `i32`→`int32`, `usize`→`uint`, etc. `i128`/`u128`
  rejected.
- **Indexing**: both 0-based. Direct.
- **`&T`** → plain value. **`&mut T`** → `var T` parameter.
- **`&[T]`** → `openArray[T]` in parameter position, `seq[T]` when owned.
  `Nim::owned()` performs that conversion.
- **Ownership/borrowck**: ignored. Nim is GC'd; for safe Rust this is sound.
- **`Option`/`Result`** → object variants in the prelude.
- **`match`** → Nim `case` where the arms are simple, `if`/`elif` when arms have
  guards or bindings.
- **Rust's expression-orientation** maps well: Nim `if`/`case` are expressions
  too, and a proc's trailing expression is its return value.

### Settled empirically (Nim 2.2.4 vs rustc 1.98.1, both run)

1. **Nim's `shr` on a signed integer is arithmetic**, matching Rust.
   `int16(-256) shr 8` = `-1` in Nim; `(-256i16) >> 8` = `-1` in Rust.
   `base16ct`'s decoder depends on this, so it maps directly with no helper.
2. **Nim's fixed-width unsigned arithmetic wraps silently**, matching Rust's
   `wrapping_*`. `uint8(200) + 100` = `44` in Nim; `200u8.wrapping_add(100)`
   = `44` in Rust. So `wrapping_add` on an unsigned type is just `+`.

3. **We model rustc's debug profile.** Rust debug builds panic on signed
   integer overflow; Nim's default build raises `OverflowDefect` on it. Those
   are the matching pair, so the runner invokes `rustc` without `-O` and `nim
   c` with its defaults, and `tests/cases/016` pins the behaviour. A Rust
   panic exits 101 where a Nim Defect exits 1, so every generated module ends
   with a handler that maps one to the other — otherwise the runner's
   exit-status comparison would be vacuous. `wrapping_*` is therefore an
   explicit operation on both sides: unsigned maps to the bare operator (item
   2), signed is routed through the unsigned view of the same width.
4. **`char` round-trips.** Rust `char` → Nim `Rune`, confirmed for ASCII and
   non-ASCII scalars in both `{}` and `{:?}`, and across `as u32`
   (`tests/cases/014`).

### Still open

5. `checked_*` and `saturating_*` are not mapped yet; they are currently
   rejected as unsupported methods rather than approximated.
6. Generics (type and const parameters), traits and trait impls, closures and
   iterator adaptors are rejected with a reason. Lifetime parameters are *not*
   a rejection: they carry no runtime meaning and Nim is GC'd, so
   `fn encode<'a>(..)` lowers fine.
7. Float formatting matches Rust for ordinary values and for `inf`/`NaN`, but
   the exponent-form thresholds have only been checked at `1e21`.
8. Flattening several files into one module can collide a crate's own
   `type Result<T>` with the builtin `Result<T, E>`. Rust kept them apart by
   module; we tell them apart by arity. That is a real difference from Rust's
   resolution and would need proper module scoping to fix.

## Testing: differential, not golden

The bar is **behavioural equivalence with rustc**, not that the output looks
plausible. For each case in `tests/cases/`:

```
rustc case.rs && ./case            > expected
rustnim case.rs -o case.nim && nim c -r case.nim > actual
diff expected actual
```

A case only counts as passing when both binaries build *and* produce identical
stdout *and* exit with the same status.

`tests/differential.rs` implements this, and checks each stage separately so a
failure says where it went wrong: `rustnim`, `rustc`, `nim`, or `diff`. Three
guards exist specifically because of how the other transpiler failed:

- `rustnim` exiting 0 while writing **no output file** is a failure.
- `rustnim` exiting 0 while writing an **empty output file** is a failure.
- An **empty corpus** is a failure, so the runner cannot pass by finding
  nothing to do.

All three have been verified by deliberately breaking the transpiler and
confirming the runner goes red.

Cases carry directives in leading `//@` comments:

| directive | meaning |
|---|---|
| `//@ reject: <substring>` | `rustnim` must *fail*, with this in its message |
| `//@ skip: <reason>` | not run; reported as skipped |
| `//@ args: <argv>` | passed to both binaries |
| `//@ stdin: <line>` | fed to both binaries |

`reject` cases are how the "fail loudly" rule is tested rather than merely
stated: `900`–`904` pin the rejections of `i128`, an unmapped standard-library
method, a float→int cast, an unimplemented format spec, and a closure.

Run one case with `RUSTNIM_CASE=005 cargo test --test differential --
--nocapture`. Nim is found at `.nim-toolchain/bin/nim` in the repository root
or any parent, or via `RUSTNIM_NIM`.

## Toolchain

- `rustc` / `cargo` 1.98.1 — system.
- Nim 2.2.4 — vendored at `.nim-toolchain/` (gitignored; downloaded from
  nim-lang.org, not installed system-wide). Binary: `.nim-toolchain/bin/nim`.

## Milestone 1

Transpile `base16ct` 1.0.0 — the crate the other transpiler failed on — and
have its decoder produce byte-identical output to the Rust original.

**Not reached.** What is reached, and measured, is `tests/cases/022`: the
`Error` enum, `decoded_len` and `decode_nibble` **verbatim from base16ct
1.0.0**, decoding lower, upper and mixed hex and both error cases, with output
byte-identical to rustc's. That covers the constant-time nibble arithmetic —
the i16 wrapping and arithmetic shift whose exact semantics the other
transpiler's float64 universal AST cannot represent at all. The loop around it
is rewritten with indexing, and the case says so.

Running the real crate now gives these diagnostics, which are the todo list:

| file | blocker |
|---|---|
| `error.rs` | `impl fmt::Display for Error`, `impl core::error::Error`, `impl From<Error> for fmt::Error` — trait impls |
| `lib.rs` | `decode_inner`: `dst.get_mut(..n)` (a mutable subslice view), `chunks_exact(2)`, `zip`, `iter_mut`, and `*dst = ..` |
| `lower.rs`, `upper.rs`, `mixed.rs` | the same, plus `encode`'s `chunks_exact_mut` |
| `display.rs` | `impl fmt::UpperHex for HexDisplay` — trait impls again |

So the remaining work is two features, not a long tail: **trait impls**, and
**iterator adaptors over slices** together with the mutable slice views they
borrow from. `mod`/multi-file and `#[cfg]` are done; pass the crate's files
together and add `--cfg feature=alloc` for the `alloc` half.