hanki

20. Test blocks and hanki test

Tests are first-class. A test "name" … end block is a top-level item, like def:

def double(x: i32) -> i32
  x + x
end

test "double doubles"
  assert!(double(2i32) == 4i32)
end

hanki test PATH parses, type-checks, and runs each test block in PATH via the bytecode interpreter, then prints a summary line N passed, M failed. The process exits non-zero if any test failed.

assert!(cond) is the built-in assertion. It is registered automatically - no import needed. On false it aborts the current test with a file:line:col pointing at the assert! call; the test runner continues with the next test. When cond is a comparison (==, !=, <, <=, >, >=), the failure captures both operands and renders each structurally via the same renderer as dbg! - named fields and variants for structs and sums - so the message shows what each side evaluated to. The test runner reports each failure as:

  FAIL  <test name> (<path>:<line>:<col>: assertion failed: <source> (left: <lhs>, right: <rhs>))

A non-comparison assert!(cond) reports the bare assertion failed form. Operand capture works on both tiers: the bytecode test runner and an AOT-compiled binary's failing assert both render both sides.

Test bodies behave like permissive actions: any effect a callee declares is allowed, and throw is permitted regardless of declared effects. The test name must be a plain string literal (no #{…} interpolation).

Property tests - a test with typed parameters

A test whose name is followed by typed parameters is a property: the runner draws each parameter from its Arbitrary instance and runs the body over many generated cases. A parameterless test is an ordinary example, exactly as above - no new keyword.

test "addition commutes"(a: i32, b: i32)
  assert!(a + b == b + a)
end

The body uses the ordinary assert!, so a failing case reports the offending values through the same operand capture as any other assertion. A parameter's type must be Arbitrary (§ extra gen / arbitrary): every fixed-width integer, the arbitrary-precision int, bool, bytes, string, decimal, rational, Option<T> / List<T> / Map<K, V> (K: Hash), and - via @derive(Arbitrary) - any struct with fields and any sum, including generic and self-recursive ones (both derive leaf-biased size-bounded generation — a sum draws its variants with Gen.frequency, the recursive ones weighted in proportion to the remaining size budget so they grade down to the non-recursive base cases as it shrinks (a strict generalization of a hard size floor) — so recursion through a recursive variant or an Option<Self> / List<Self> field shrinks the size budget to a base case and terminates; the size-aware Option / List instances draw None / the empty list at the size floor; a generic type's params are each bounded by Arbitrary). Recursion terminates whether a type reaches itself directly (a recursive variant, an Option<Self> / List<Self> field) or by mutual / indirect recursion through another type (A whose field holds a B that holds an A): the deriver walks the module's type-reference graph and shrinks the size budget at every field that reaches the type being derived, so the budget halves around the cycle and bottoms out at the Option / List size floor. (int draws across the i64 range, bytes up to a 16-byte length, string as up to 16 full-Unicode codepoints, decimal as mantissa × 10^-scale, rational as an exact num/denom, and Map as up to 16 drawn entries — so every builtin type is Arbitrary.)

Generation is pure: the runner threads a seeded ChoiceSource, so a property is reproducible by default (the seed derives from the test name) and there is no [random] effect infecting generators. An optional @property(...) attribute tunes it:

@property(cases: 1000, seed: 42)
test "addition is associative"(a: i32, b: i32, c: i32)
  assert!((a + b) + c == a + (b + c))
end

A failing property reports its counterexample - the sampled parameters of the failing case, each rendered name = value and joined - on the text FAIL line and in the --format=json envelope. The counterexample is shrunk first: integrated shrinking minimizes the recorded choice sequence (deleting choices, then reducing each toward zero) and re-runs the same generator, so the reported input is a small one that still fails rather than the raw random case. Because a generator is a deterministic function of its choices, this needs no per-type shrinker; and because the built-in bounded draws (int_range, list length, one_of and its weighted sibling frequency) use a monotone choice→value encoding - a smaller choice yields a smaller value - a wide scalar shrinks to its true minimum, not merely to a smaller reproducing value.

The shrunk failing input is also persisted to a corpus at .hanki/corpus/<test-name>.txt (the minimized choice sequence). On the next run the corpus is replayed before any new exploration, so a regression is caught immediately rather than only when random sampling happens to re-find it; once the corpus entry passes (the bug is fixed) exploration resumes normally. On by default; --no-corpus opts out for a stateless run.

Fuzzing - @fuzz and hanki fuzz

A property tests an assertion over many inputs; a fuzz target tests totality - that code does not crash on any input. Mark a property test @fuzz to make it a fuzz target; the body need not assert anything - exercising the code under test is enough, since a crash on any input fails the run:

def sum(xs: List<i32>) -> i32
  xs.fold(0i32, |acc, x| acc + x)
end

@fuzz test "sum is total"(xs: List<i32>)
  _ = sum(xs)
end

A @fuzz target is a property test in every mechanical respect - typed parameters drawn from their Arbitrary instances (§ extra gen / arbitrary), pure seeded generation, integrated shrinking, and the same .hanki/corpus/ replay - with two differences:

hanki fuzz PATH parses, type-checks, and runs each @fuzz target via the bytecode interpreter, drawing --cases inputs per target (default 1000; trades run time for coverage) and failing the run on the first that crashes - reporting the shrunk offending input exactly as a property counterexample, on the text FAIL line and in the --format=json hanki-test-v1 envelope. A target's seed derives from its name, so a fuzz failure is reproducible and its corpus entry replays first on the next run. Alternatively, --time <SECONDS> bounds each target by wall clock instead of a fixed count - it explores until the budget elapses (taking precedence over --cases), so a fuzz run scales to the time you give it; it cannot combine with --deterministic (a time budget is non-reproducible). It takes the same other orthogonal knobs as hanki test: --stdlib (also fuzz stdlib @fuzz targets - off by default), --no-corpus, --format, --deterministic/--seed, and the --max-steps/--max-bytes per-case budgets (a budget fault fails the target, never shrunk).

hanki fuzz also takes --deny/--allow (§23): with the same declared-surface semantics as every other command, a @fuzz target whose reachable effect surface violates the allowance is refused before the run. A finer oracle that faults a target the moment it performs an undeclared effect mid-case (rather than on its declared surface) is planned.

Doctests - runnable examples in documentation comments

A documentation comment (§2) can carry runnable examples. A fenced code block inside it - opened with a bare or hanki and closed with - is compiled and run as a synthesized test` in the module that defines the documented item. This keeps documentation honest: an example that no longer type-checks, or whose result changed, fails the test run.

# Returns the wrapped value, or `default` when the option is `None`.
#
# ```
# Some(7i32).unwrap_or(0i32) => 7i32
# n: Option<i32> = None
# n.unwrap_or(9i32)          => 9i32
# ```
def unwrap_or(self, default: T) -> T
  ...
end

Presence gate (core stdlib). Every exported, callable core symbol - a top-level def (pure or action), an impl/trait method, an effect op, including an @intrinsic - must carry a runnable example or an explicit exemption. The exemption is a # @no-doctest: <reason> directive line inside the symbol's doc comment, with a non-empty reason; it is reserved for the few symbols an EXPR => VALUE assertion genuinely cannot illustrate (a side-effecting OS primitive like sys.stdout_write!, a runtime-coupled loader like module.load!). A bare @no-doctest with no reason does not exempt. The bar is enforced two ways: a compiler test over the embedded core stdlib, and hanki doc --check (below). Actor on handlers are covered too - they are the actor's message protocol, and a handler's doc example is an ordinary self-contained program (it spawns the actor itself). Type/struct/actor declarations themselves are out of scope. The extra tier is held to a deliberately weaker per-module bar: a module with any exported callable symbol must carry at least one runnable example or one @no-doctest: <reason> exemption somewhere among its symbols - enough to forbid a silently undocumented-and-unjustified module without mandating an example on every symbol as core does (a compiler test enforces it; a module escalates to full per-symbol coverage as its examples get written).

Generated reference - hanki doc

A documentation comment is attached to the item it precedes (the same adjacency rule doctests use), so its prose is structured data keyed to the symbol - not merely a doctest source. A leading comment detached from the first item by a blank line documents the module. hanki doc FILE.hk renders this as Markdown on stdout: one section per documented item - the item's signature in a fenced block, then the comment's prose, with the comment's own examples surviving as ordinary code blocks. Each heading carries a stable {#anchor} for cross-references. Pass a symbol to narrow to one item - hanki doc FILE.hk map (bare name) or hanki doc FILE.hk Option.map (qualified Type.method). A bare argument that matches no file on disk but names a **baked stdlib module** renders that module's documentation instead - hanki doc list, hanki doc list map (a .hk suffix is tolerated) - so an agent recalls a stdlib surface (append vs a hallucinated push) with no source file to point at; a real file always shadows a same-named module. Undocumented items still render (signature only), so the output doubles as an API surface; use/open imports are omitted. Point it at a **project directory** instead of a file - hanki doc DIR - to render the whole project: the manifest entry is resolved and every reachable module is emitted in the workspace's topological order (dependencies first), Text as one aggregated Markdown document (each module under an # <path> heading), --format=json` as a JSON array of per-file envelopes.

hanki doc --check FILE lints instead of rendering: it prints nothing on success and exits non-zero listing every exported callable symbol that has neither a runnable example nor a # @no-doctest: <reason> exemption (the presence gate above). It works on any file, so a project can enforce the example bar on its own public surface, not just the stdlib.

hanki doc FILE --format=json emits the same surface as data - one hanki-doc-v1 envelope with each symbol's signature, effect row, doc prose, and doctest presence (§23 has the shape).

Coverage - hanki test --coverage / hanki run --coverage

Both subcommands accept --coverage to instrument the bytecode interpreter and report which source lines and branches executed. After the run, a per-file terminal table is printed and an lcov report is written to target/coverage/lcov.info (path hardcoded - editor extensions and CI configure once; no flag).

$ hanki test --coverage
2 passed, 0 failed
File             Lines    Hit   Line%   Branch  BrHit     Br%
-------------------------------------------------------------
src/main.hk        4      3   75.0%        2      1   50.0%
-------------------------------------------------------------
TOTAL                4      3   75.0%        2      1   50.0%

lcov report: target/coverage/lcov.info

A branch direction whose immediate block always diverges via crash! (§6) is exempt from the branch denominator. Because crash! aborts, no passing test can exercise that direction, so counting it would put 100% branch coverage out of reach for code with a deliberate else crash!(...) guard. Only the immediate block is inspected: a crash! reached only after an intervening branch (e.g. one whose message is itself computed with an if) is conservatively still counted.

Scope: line and branch coverage, bytecode interpreter only, presence-only hits, no per-test reports. AOT-tier instrumentation, HTML, and FN/FNDA lcov records are not yet implemented.