hanki

20. Test blocks and hanki test

Tests are part of the language. A test "name" … end block is a top-level item, like def:

def double(x: i32) -> i32
  x + x
end

test "double doubles"
  assert!(double(2i32) == 4i32)
end

hanki test PATH parses, type-checks, and runs each test block in PATH via the bytecode interpreter, then prints a summary line N passed, M failed. The process exits non-zero if any test failed.

assert!(cond) is the built-in assertion, registered automatically with no import needed. Its effect row is [Crash], the same atom crash! takes: a failed assertion aborts, and §6's rule is that divergence is an effect. A test context permits it without listing it, and assertions inside tests are unaffected, while an action that asserts declares [Crash] like any other caller of a diverging primitive. On false it aborts the current test with a file:line:col pointing at the assert! call, and the test runner continues with the next test. Where cond is a comparison, ==, !=, <, <=, > or >=, the failure captures both operands and renders each structurally through the same renderer as dbg!, with named fields and variants for structs and sums, and the message shows what each side evaluated to. The test runner reports each failure as:

  FAIL  <test name> (<path>:<line>:<col>: assertion failed: <source> (left: <lhs>, right: <rhs>))

A non-comparison assert!(cond) reports the bare assertion failed form. Operand capture works on both tiers: the bytecode test runner and an AOT-compiled binary's failing assert both render both sides.

Test bodies behave like permissive actions: any effect a callee declares is allowed, and throw is permitted whatever the declared effects. The test name must be a plain string literal, with no #{…} interpolation.

Property tests: a test with typed parameters

A test whose name is followed by typed parameters is a property: the runner draws each parameter from its Arbitrary instance and runs the body over many generated cases. A parameterless test is an ordinary example, as above, and there is no new keyword.

test "addition commutes"(a: i32, b: i32)
  assert!(a + b == b + a)
end

The body uses the ordinary assert!, and a failing case reports the offending values through the same operand capture as any other assertion. A parameter's type must be Arbitrary (core gen and arbitrary): every fixed-width integer, the arbitrary-precision int, bool, bytes, string, decimal, rational, Option<T>, List<T> and Map<K, V> with K: Hash, and, through @derive(Arbitrary), any struct with fields and any sum, generic and self-recursive ones included. Both derive leaf-biased size-bounded generation: a sum draws its variants with Gen.frequency, the recursive ones weighted in proportion to the remaining size budget so that they grade down to the non-recursive base cases as it shrinks, a strict generalization of a hard size floor. Recursion through a recursive variant, or through an Option<Self> or List<Self> field, therefore shrinks the size budget to a base case and terminates, and the size-aware Option and List instances draw None and the empty list at the size floor. A generic type's params are each bounded by Arbitrary. Recursion terminates whether a type reaches itself directly, a recursive variant or an Option<Self> or List<Self> field, or by mutual or indirect recursion through another type, an A whose field has a B with an A in it. The deriver walks the module's type-reference graph and shrinks the size budget at every field that reaches the type being derived, the budget halves around the cycle, and it bottoms out at the Option and List size floor. int draws across the i64 range, bytes up to a 16-byte length, string as up to 16 full-Unicode codepoints, decimal as mantissa × 10^-scale, rational as an exact num/denom, and Map as up to 16 drawn entries, and every builtin type is therefore Arbitrary.

Generation is pure: the runner threads a seeded ChoiceSource, a property is reproducible by default, the seed deriving from the test name, and no [random] effect infects generators. An optional @property(...) attribute tunes it:

@property(cases=1000, seed=42)
test "addition is associative"(a: i32, b: i32, c: i32)
  assert!((a + b) + c == a + (b + c))
end

The options form a comma-separated list with no trailing comma.

Each key may appear at most once.

A failing property reports its counterexample, the sampled parameters of the failing case, each rendered name = value and joined, on the text FAIL line and in the --format=json envelope. The counterexample is shrunk first: integrated shrinking minimizes the recorded choice sequence, deleting choices and then reducing each toward zero, and re-runs the same generator, and the reported input is therefore a small one that still fails in place of the raw random case. A generator is a deterministic function of its choices, and this needs no per-type shrinker. The built-in bounded draws, int_range, list length, one_of and its weighted sibling frequency, use a monotone choice-to-value encoding where a smaller choice yields a smaller value, and a wide scalar therefore shrinks to its true minimum and not merely to a smaller reproducing value.

The shrunk failing input is also persisted to a corpus at .hanki/corpus/<test-name>.txt, holding the minimized choice sequence. On the next run the corpus is replayed before any new exploration, and a regression is caught immediately in place of only when random sampling happens to re-find it; once the corpus entry passes, the bug being fixed, exploration resumes normally. It is on by default, and --no-corpus opts out for a stateless run.

Fuzzing - @fuzz and hanki fuzz

A property tests an assertion over many inputs. A fuzz target tests totality, that code does not crash on any input. Mark a property test @fuzz to make it a fuzz target. The body need not assert anything, exercising the code under test being enough, since a crash on any input fails the run:

def sum(xs: List<i32>) -> i32
  xs.fold(0i32, |acc, x| acc + x)
end

@fuzz test "sum is total"(xs: List<i32>)
  _ = sum(xs)
end

A @fuzz target is a property test in every mechanical respect, with typed parameters drawn from their Arbitrary instances (core gen and arbitrary), pure seeded generation, integrated shrinking, and the same .hanki/corpus/ replay, and two differences:

hanki fuzz PATH parses, type-checks and runs each @fuzz target through the bytecode interpreter, drawing --cases inputs per target, 1000 by default, trading run time for coverage, and failing the run on the first that crashes. It reports the shrunk offending input as a property counterexample, on the text FAIL line and in the --format=json hanki-test-v1 envelope. A target's seed derives from its name, a fuzz failure is therefore reproducible, and its corpus entry replays first on the next run. --time <SECONDS> bounds each target by wall clock in place of a fixed count, exploring until the budget elapses and taking precedence over --cases, and a fuzz run therefore scales to the time you give it. It cannot combine with --deterministic, a time budget being non-reproducible. It takes the same other orthogonal knobs as hanki test: --stdlib, which also fuzzes stdlib @fuzz targets and is off by default, --no-corpus, --format, --deterministic and --seed, and the --max-steps and --max-bytes per-case budgets, a budget fault failing the target and never being shrunk.

hanki fuzz also takes --deny and --allow (§23), with the same declared-surface semantics as every other command: a @fuzz target whose reachable effect surface violates the allowance is refused before the run. A finer oracle that faults a target the moment it performs an undeclared effect mid-case, in place of on its declared surface, is planned.

Doctests: runnable examples in documentation comments

A documentation comment (§2) can hold runnable examples. A fenced code block inside it, opened with a bare or hanki and closed with , is compiled and run as a synthesized test` in the module that defines the documented item. An example that no longer type-checks, or whose result changed, therefore fails the test run.

# Returns the wrapped value, or `default` when the option is `None`.
#
# ```
# Some(7i32).unwrap_or(0i32) => 7i32
# n: Option<i32> = None
# n.unwrap_or(9i32)          => 9i32
# ```
def unwrap_or(self, default: T) -> T
  ...
end

The stdlib has a presence gate. Every exported, callable stdlib symbol, a top-level def pure or action, an impl or trait method, an effect op, an @intrinsic included, must have a runnable example or an explicit exemption. The exemption is a # @no-doctest: <reason> directive line inside the symbol's doc comment, with a non-empty reason, and it is reserved for the few symbols an EXPR => VALUE assertion cannot illustrate: a side-effecting OS primitive like sys.stdout_write!, or a runtime-coupled loader like module.load!. A bare @no-doctest with no reason does not exempt. Two kinds of symbol are outside the gate and need no exemption line: a _-prefixed name, private to its module, and a program's entry point, a top-level main!, since nothing calls it but the runtime and the only example of it would be running the program, which is what the program is. That exemption is keyed on the name at top level and not on the manifest's declared entry, and a member called main! is an ordinary method and still gated. The bar is enforced two ways, by a compiler test over the embedded core stdlib and by hanki doc --check, below. Actor on handlers are covered too, being the actor's message protocol, and a handler's doc example is an ordinary self-contained program that spawns the actor itself. Type, struct and actor declarations themselves are out of scope. The bar is the same for both tiers, extra included: hanki doc --check means the same thing wherever it is pointed.

Generated reference - hanki doc

A documentation comment is attached to the item it precedes, on the same adjacency rule doctests use, and its prose is therefore structured data keyed to the symbol and no mere doctest source. A leading comment detached from the first item by a blank line documents the module. hanki doc FILE.hk renders this as Markdown on stdout: one section per documented item, the item's signature in a fenced block and then the comment's prose, with the comment's own examples surviving as ordinary code blocks. Each heading has a stable {#anchor} for cross-references. Pass a symbol to narrow to one item: hanki doc FILE.hk map for a bare name, or hanki doc FILE.hk Option.map for a qualified Type.method. A bare argument that matches no file on disk and names a **baked stdlib module** renders that module's documentation instead, hanki doc list or hanki doc list map, a .hk suffix being tolerated, and an agent can therefore recall a stdlib surface, append against a hallucinated push, with no source file to point at; a real file always shadows a same-named module. An unambiguous public type name also resolves to the module that documents it, as in hanki doc UnixStream. A shared type name reports the qualified candidates and requires the module form, such as hanki doc aead Key. Undocumented items still render, signature only, and the output therefore doubles as an API surface, and use and open imports are omitted. Point it at a **project directory** in place of a file, hanki doc DIR, to render the whole project: the manifest entry is resolved and every reachable module is emitted in the workspace's topological order, dependencies first, as one aggregated Markdown document with each module under an # <path> heading, or under --format=json` as a JSON array of per-file envelopes.

hanki doc --check FILE lints in place of rendering: it prints nothing on success, and exits non-zero listing every exported callable symbol that has neither a runnable example nor a # @no-doctest: <reason> exemption, the presence gate above. It works on any file, and a project can therefore enforce the example bar on its own public surface, and not on the stdlib alone. In this repo the contrib/ packages are gated on it and examples/ is not: a contrib package is a library surface a caller reaches for without opening the file, where a missing example costs the reader something, while an example project is read whole and its main! documents itself by being the thing the reader came for.

hanki doc FILE --format=json emits the same surface as data: one hanki-doc-v1 envelope with each symbol's signature, effect row, doc prose and doctest presence (§23 gives the record).

hanki doc --find TERM searches every baked module and every seeded contrib package, symbol names and doc prose, case-insensitive substring, and prints module-qualified hits with their signatures, name matches and prose matches in separate sections, a name hit being a likely answer and a prose hit a lead. A contrib hit is marked # contrib, the package needing hanki add before the symbol resolves, and hanki doc <package> reads one before it is a dependency. It searches the collected doc surface and not the text, and it therefore sees impl-block members, which a line-anchored grep over the sources does not. The negative matters most: no hits is an authoritative statement that neither tier has this, where proving that by grep means guessing every spelling it might have been given. The miss names both counts, which lets a reader tell a tier that was searched and came back empty from one that was not searched at all. A miss exits non-zero in text mode and emits an empty hits array at exit 0 under --format=json, the same convention the symbol filter uses.

hanki doc with no argument is the index: every baked stdlib module, grouped by tier, one line each, the module name and the first sentence of its header comment, clipped to a terminal width. It is how a reader who does not already know a module's name reaches one at all, and hanki doc <name> then renders it in full. --format=json answers the same as a hanki-doc-index-v1 envelope (§23). Run inside a project it adds a third section for that package's own modules, taken from the manifest's exports, its declared public surface, and not from a walk over whatever .hk files are lying about. They are marked project in the envelope's per-entry tier field, and a tool therefore tells them apart with no second record. A package declaring no exports prints no such section. This does make the bare command depend on the working directory, which hanki test and hanki check already do, and the labelled sections are what keep provenance unambiguous. hanki doc <name> resolves past the stdlib into the project's declared dependencies, in that order, a stdlib module first and then a dependency alias, and hanki doc tui therefore reads the package use tui binds, and hanki doc tui Plot one item of it, without the reader knowing where that source sits on disk. A name in neither namespace says so, naming both and not the stdlib alone.

A pointer must sit downstream of where its question forms. A doc gate can check that a symbol is documented, and it cannot check that the documentation is where the reader will be standing. Readers enter a module through its verbs, arriving to look for the thing that does something, and an answer parked on a type's page, or in a neighbouring module's chapter, is not read even where it is correct and on the same rendered page. Two cases made the rule: aead's to_bytes, documented on Key about a hundred rendered lines above new_key!, where a reader who entered at the mint call never scrolled back and built a workaround around a capability that was there; and stopping a subprocess, answered in the actors material with nothing in process saying so.

A module doc therefore has, near its top, a short block naming the two or three questions that predictably land on it and are answered elsewhere, of the form "arriving here for X? it is in Y", and a symbol whose obvious follow-up question belongs to another symbol answers it at the call site the reader reached first and not only where it structurally belongs. This is prose work per module and no mechanism: the whole cost is asking, once per module, what someone arrives here wanting that this file does not answer. Resist building a cross-reference framework for it.

Coverage - hanki test --coverage / hanki run --coverage

Both subcommands accept --coverage, which instruments the bytecode interpreter and reports which source lines and branches executed. After the run, a per-file terminal table is printed and an lcov report is written to target/coverage/lcov.info. That path is hardcoded: editor extensions and CI configure once, and there is no flag. That one file measures one program, and a package declaring several is refused without --program <name> in place of reporting whichever ran last, the same file-stem selector and the same refusal hanki run and hanki build use for the same situation. Without --coverage, hanki test on such a package runs every declared root's suite as before.

$ hanki test --coverage
2 passed, 0 failed
File             Lines    Hit   Line%   Branch  BrHit     Br%
-------------------------------------------------------------
src/main.hk        4      3   75.0%        2      1   50.0%
-------------------------------------------------------------
TOTAL                4      3   75.0%        2      1   50.0%

lcov report: target/coverage/lcov.info

A branch direction whose immediate block always diverges through crash! (§6) is exempt from the branch denominator. crash! aborts, no passing test can exercise that direction, and counting it would put 100% branch coverage out of reach for code with an else crash!(...) guard. Only the immediate block is inspected: a crash! reached only after an intervening branch, one whose message is itself computed with an if for instance, is conservatively still counted.

Scope: line and branch coverage, bytecode interpreter only, presence-only hits, no per-test reports. AOT-tier instrumentation, HTML, and FN/FNDA lcov records are not yet implemented.

Allocation profile - hanki run --profile-allocs / hanki build --profile-allocs

hanki check --explain-copies names the field writes the uniqueness analysis could not prove unshared (§13). It reports what a write may cost. hanki run --profile-allocs reports what one run did cost: it counts the containers each write rebuilt and prints one line per write, most allocations first.

$ hanki run src/main.hk --profile-allocs
copy-site allocations: 15 across 2 sites
  12  src/main.hk:19:3
   3  src/main.hk:41:5

The two read together. A profiled count is keyed to the same write span an H0564 names, which puts a price on a lint that has none on its own. The pairs a reader is looking for are a write the lint flags that never appears in the profile, whose runtime check found a unique root and allocated nothing, and a write with a small static message and a large count, which is a rebuild inside a hot loop.

Scope: bytecode and AOT tiers, field-write copy sites only. The report is a terminal table, with no machine-readable envelope.