Test Infrastructure

INFO

This article serves as an invitation for sharing ideas to improve our test infrastructure, feel free to contact us on [Discord][discord-url].

In Oxc, correctness and reliability are taken extremely seriously.

We spend a great deal of time strengthening the test infrastructure to prevent problems from propagating to downstream tools.

Parser

Conformance

Parser tests from Test262, Babel, and TypeScript are used to test JavaScript, TypeScript, and JSX syntax.

For Test262, all stage 4 and regular expression tests are included.

All conformance results are stored in a snapshot file for tracking changes:

• test262.snap.
• babel.snap.
• typescript.snap.

All syntax errors are written to these snapshot files for diffing changes.

Fuzzing

To ensure that the parser does not panic when encountering random data, three fuzzers are used:

1. cargo fuzz for sending random bytes to the parser.
2. shift-fuzzer-js by bakkot for producing random but valid ASTs.
3. Automated-Fuzzer by qarmin, which actively reports crashes.

Memory Safety

Oxc uses an arena allocator based around bumpalo as the memory allocator for its AST, and other data. None of the AST node types have a Drop implementation. This is enforced at compile time by Oxc's allocator, which causes a compile-time error if any code attempts to allocate types in the arena which are Drop.This statically ensures that types which own heap-allocated data cannot be stored in the arena, which would result in memory leaks.

Unsafe code

Oxc uses unsafe code for performance optimizations. We aim to contain unsafe to within self-contained data structures which present safe APIs externally. Miri is run on the crates containing these structures on every PR.

Linter

Snapshot Diagnostics

All linter diagnostics are written to a snapshot file for testing against regressions.

For example:

 ⚠ typescript-eslint(adjacent-overload-signatures): All "foo" signatures should be adjacent.
  ╭─[adjacent_overload_signatures.tsx:3:18]
2 │         function foo(s: string);
3 │         function foo(n: number);
  ·                  ───
4 │         type bar = number;
5 │         function foo(sn: string | number) {}
  ·                  ───
6 │       }
  ╰────
``````javascript
let sourceText = "foo";
let printed = tool(sourceText);
let printed2 = tool(printed);
assert(printed == printed2);
``````json
"devDependencies": {
  "@aashutoshrathi/word-wrap": "latest",
  "@actions/http-client": "latest",
  "@adobe/css-tools": "latest",
  "@alloc/quick-lru": "latest",
 ...
  "zip-stream": "latest",
  "zod": "latest",
  "zone.js": "latest",
  "zustand": "latest"
}
``````javascript
import test from "node:test";
import assert from "node:assert";
test("@aashutoshrathi/word-wrap", () => import("@aashutoshrathi/word-wrap").then(assert.ok));
test("@actions/http-client", () => import("@actions/http-client").then(assert.ok));
test("@adobe/css-tools", () => import("@adobe/css-tools").then(assert.ok));
test("@alloc/quick-lru", () => import("@alloc/quick-lru").then(assert.ok));
...
test("zod", () => import("zod").then(assert.ok));
test("zone.js", () => import("zone.js").then(assert.ok));
test("zustand", () => import("zustand").then(assert.ok));
test("zwitch", () => import("zwitch").then(assert.ok));