fs

stdlib/extra/fs.hk: filesystem reads and writes.

A pure-Hanki face over the sys native seam: these functions bear no @intrinsic, they delegate to the raw sys primitives, and the filesystem effect bubbles up automatically: read!/open_file! charge [fs_read], write! charges [fs_write] (the [fs] alias expands to both). Policy and ergonomics belong here and never in sys.

read!/write! return Result over a sys.FsFailure: the path the call named beside a specific kind (NotFound / PermissionDenied / NotUtf8 / Other / AlreadyExists / NotADirectory), and no collapse of every failure into None or false.

What a missing path means here

The module runs on one rule, and it turns on what the call is for:

• Predicates (exists!, is_dir!): absence is the answer, and a missing path is Ok(false). Only a real stat failure is Err.
• Retrievals (read!, open_file!, list_dir!, metadata!): absence is a failure to retrieve, and a missing path is Err(NotFound) with the path attached.

Written down because it has been re-derived from scratch twice, each time by someone deciding whether a new call should answer Ok(None). It should not: Err also names which path was missing, which is what a caller statting twenty files needs and None cannot give.

Runs on both tiers (bytecode and LLVM AOT): each runtime implements the raw sys file primitives over the shared OS layer, and an AOT-built program does real filesystem I/O.

No hermetic test block belongs here: every function needs real, non-deterministic OS state, the delegations being the seam itself and walk! needing a tree on disk to walk, which leaves nothing here assertable in the pure stdlib test pack. Coverage is end-to-end through the CLI integration tests and example programs.

Arriving here for something this module does not have

• Manipulating a path. extra/path does it lexically (join, dirname, basename, normalize, split_ext) with no I/O and no effect row, which makes it work from a pure def.
• Filtering a walk. walk! hands back every file beneath a root; extra/glob is the filter for it (compile(pattern), then pattern.matches?(path)).
• A temporary directory. env.temp_dir! names one (TMPDIR, else /tmp) and creates nothing. Making a directory under it is mkdir! here. The two are one word apart, which is worth saying out loud.

Permissions are here: write_with_permissions! creates a file at the mode asked for and set_permissions! changes an existing one. A secret wants the first, plain write! landing at whatever the process umask gives it, which is world-readable on a common box.

read!

def read!(path: string) -> Result<string, sys.FsFailure> [fs_read]

Read the entire file at path as a UTF-8 string. @no-doctest: reads a real file; needs a real path, and cannot assert in a doctest

read_bytes!

def read_bytes!(path: string) -> Result<bytes, sys.FsFailure> [fs_read]

Read the entire file at path as raw bytes, for content that is not UTF-8 text: a PDF, an image, an archive. read! decodes as UTF-8 and fails on anything else, which left copy! able to move a file the module could not read.

The handle is closed eagerly and never left to the last handle drop: a caller reading many files in a loop would otherwise hold every descriptor until its handle went out of scope. @no-doctest: reads a real file; needs a real path, and cannot assert in a doctest

canonicalize!

def canonicalize!(path: string) -> Result<string, sys.FsFailure> [fs_read]

Resolve path to its absolute, symlink-free form (the OS realpath). Ok(resolved) when path exists, Err(FsFailure) otherwise. @no-doctest: resolves a real path; needs one that exists, and cannot assert in a doctest

write!

def write!(path: string, content: string) -> Result<(), sys.FsFailure> [fs_write]

Write content to the file at path, creating or replacing it. @no-doctest: writes the filesystem; side-effecting, and cannot assert in a doctest

write_bytes!

def write_bytes!(path: string, content: bytes) -> Result<(), sys.FsFailure> [fs_write]

Write raw content to the file at path, creating or replacing it: the write half of read_bytes!, for content that is not UTF-8 text. A program that could read a PDF could not write one back.

write! is the string face and is unchanged: bytes and string are distinct types, and one function cannot serve both without a union that would make every call site say which it meant anyway. @no-doctest: writes the filesystem; side-effecting, and cannot assert in a doctest

Access

type Access
  NoAccess
  Read
  Write
  Execute
  ReadWrite
  ReadExecute
  WriteExecute
  AllAccess
end

What one class of user may do with a file: the eight-value lattice, each combination named and never assembled from booleans, which leaves no invalid state to construct and no boolean triple at the call site.

NoAccess and not None, which would read as Option's to anyone skimming a call.

Permissions

struct Permissions
  owner: Access
  group: Access
  other: Access
end

The three POSIX classes, one Access each.

Hanki has no octal literal, which suits this: 0600 is a number most reviewers read by counting on their fingers, and what a permission is, who may do what, is what this says.

Permissions(owner=ReadWrite, group=NoAccess, other=NoAccess).mode => 384

impl Access

bits

prop bits(self) -> i32

The three-bit POSIX pattern this access is, rwx high to low.

NoAccess.bits => 0
Read.bits => 4
ReadWrite.bits => 6
AllAccess.bits => 7

from_bits

def from_bits(pattern: i32) -> Access

The access a three-bit POSIX pattern names, the inverse of bits. Only the low three bits are read, and a class can be shifted out of a whole mode without masking it first.

Access.from_bits(0i32) => NoAccess
Access.from_bits(6i32) => ReadWrite
Access.from_bits(15i32) => AllAccess

impl Permissions

mode

prop mode(self) -> i32

The nine-bit POSIX mode, owner in the high three bits. This is the only place in the module that thinks in numbers.

owner_read_write().mode => 384
shared_read().mode => 420
shared_executable().mode => 493

from_mode

def from_mode(mode: i32) -> Permissions

The permissions a nine-bit POSIX mode names, the inverse of mode. Higher bits are ignored and never rejected, setuid and setgid and sticky and the file-type bits a raw stat mode has among them. This reads the three access classes, and a mode straight from metadata! has those others set.

Permissions.from_mode(384i32) => owner_read_write()
Permissions.from_mode(420i32) => shared_read()
Permissions.from_mode(493i32) => shared_executable()

impl Eq<Access>

eq?

def eq?(self, other: Self) -> bool

Two accesses are equal when they are the same three bits, which over a closed eight-value lattice is the same thing as being the same variant.

Hand-written and not derived: an on-demand synthesised impl lands at module end, which is the wrong chunk of the merged stdlib module.

Access.from_bits(6i32) == ReadWrite => true

impl Eq<Permissions>

eq?

def eq?(self, other: Self) -> bool

Two permission sets are equal when all three classes are.

owner_read_write() == Permissions(owner=ReadWrite, group=NoAccess, other=NoAccess) => true

impl Display<Permissions>

to_string

def to_string(self) -> string

The rw------- form ls -l prints, which reads the way the tool the reader will check it with does.

owner_read_write().to_string() => "rw-------"
shared_executable().to_string() => "rwxr-xr-x"

_triple

def _triple(a: Access) -> string

_flag

def _flag(bits: i32, bit: i32, letter: string) -> string

ownerreadwrite

def owner_read_write() -> Permissions

0600 - the owner reads and writes, nobody else sees it. The mode a persisted secret wants.

owner_read_write().to_string() => "rw-------"

owner_all

def owner_all() -> Permissions

0700 - the owner does everything, nobody else. What a private directory wants, since a directory needs the execute bit to be entered at all.

owner_all().to_string() => "rwx------"

read_only

def read_only() -> Permissions

0444 - everyone reads, nobody writes.

read_only().to_string() => "r--r--r--"

shared_read

def shared_read() -> Permissions

0644 - the owner writes, everyone reads. The ordinary file default.

shared_read().to_string() => "rw-r--r--"

shared_executable

def shared_executable() -> Permissions

0755 - the owner writes, everyone reads and runs. A program, or a directory others may enter.

shared_executable().to_string() => "rwxr-xr-x"

writewithpermissions!

def write_with_permissions!(path: string, content: string, permissions: Permissions) -> Result<(), sys.FsFailure> [fs_write]

Write content to path at permissions, creating or replacing the file.

This is the call a secret wants. write! creates at the process umask, which is 0644 on a common 022 box, and a key written with it is world-readable from the instant it exists; tightening it afterwards with set_permissions! leaves that instant open. Here the file is opened with the mode, and never exists at a wider one.

The mode is applied and not subjected to the umask: it goes on the open and is re-applied after it, and asking for shared_read() under a 077 umask therefore gives rw-r--r-- and not the rw------- the umask alone would leave. The umask is a default; this is a request.

The one case the window cannot be closed for is a path that already exists: the OS ignores the open mode for an existing inode, and the file retains its current mode until the write finishes and the mode is set. Write a secret to a fresh path, or to a temporary one and rename it.

Refuses with Unsupported on a non-POSIX host in place of writing the file and leaving the mode unapplied: the mode is what the call is for. @no-doctest: writes the filesystem; side-effecting, and cannot assert in a doctest

writebyteswith_permissions!

def write_bytes_with_permissions!(path: string, content: bytes, permissions: Permissions) -> Result<(), sys.FsFailure> [fs_write]

write_with_permissions! for raw bytes. A secret is as likely to be a key file or a binary token as it is to be text, and the mode is what makes it a secret. The bytes face gets the same guarantee, and no caller has to write wide and narrow afterwards.

Everything write_with_permissions! says about what the applied mode costs and gives, the umask and the window and the existing-path case and the non-POSIX refusal, applies here unchanged. @no-doctest: writes the filesystem; side-effecting, and cannot assert in a doctest

set_permissions!

def set_permissions!(path: string, permissions: Permissions) -> Result<(), sys.FsFailure> [fs_write]

Set the mode of the existing file or directory at path to permissions. No umask is involved: chmod(2) sets what it is given.

For a file that already has a secret in it this is the best available answer and still not a good one: the secret was readable between the write and this call. Reach for write_with_permissions! when the file is yours to create. @no-doctest: writes the filesystem; side-effecting, and cannot assert in a doctest

createtempdir!

def create_temp_dir!(parent: string, prefix: string, permissions: Permissions) -> Result<string, sys.FsFailure> [fs_write]

Create a fresh, private temporary directory under parent, and hand back its path. prefix leads the generated name, which lets a stray directory be traced back to what made it.

The guarantee, and the reason this is in the stdlib and not in every caller: the directory did not exist before this call, and at owner_all() only its owner can enter it. Both halves matter and the first is the hard one, checking whether a name is free and then creating it leaving a window another process can win. mkdir(2) is atomic and fails when the name is taken, and this therefore creates and retries, which is mktemp's own algorithm. Each name draws 64 bits from the CSPRNG.

permissions is almost always owner_all(): a directory needs the execute bit to be entered at all, and owner_read_write() would make one nobody can use, the caller included.

env.temp_dir! is the other half of the pair and a different thing: it names the system temporary directory (TMPDIR, else /tmp) and creates nothing. It is what a caller passes here as parent.

Nothing removes it. Hanki has no defer and no destructor, and the caller therefore removes it with delete!(path, true) when done. A scoped form taking a closure is not offered: it could not run its cleanup when the body crashes, a crash unwinding past it to actor death, and an API that promises cleanup and skips it unannounced on the one path where a caller most want it is worse than one that never promised. @no-doctest: creates a directory; side-effecting, and cannot assert in a doctest

File

File, or fs.File, is a runtime-managed native resource handle (HANKI.md §4): live native state the per-actor memory manager owns, released when the last handle drops. It is single-owner - never copied, moved across a send - and has no fields of its own, so its methods are its whole surface. A handle is minted by an API that opens one; it is never constructed.

open_file!

def open_file!(path: string) -> Result<File, sys.FsFailure> [fs_read]

Open the file at path for reading, returning a File handle. The handle's descriptor is closed when its last handle goes, or eagerly via File.close!. (open_file! and never open!: open is a keyword.) @no-doctest: opens a real file; needs one on disk, and cannot assert in a doctest

exists!

def exists!(path: string) -> Result<bool, sys.FsFailure> [fs_read]

Whether path exists (following symlinks). A missing path is Ok(false); only a real stat failure (e.g. a permission error) is Err. @no-doctest: stats the filesystem; needs a real path, and cannot assert in a doctest

is_dir!

def is_dir!(path: string) -> Result<bool, sys.FsFailure> [fs_read]

Whether path exists and is a directory. A missing path or a non-directory is Ok(false); a real stat failure is Err. @no-doctest: stats the filesystem; needs a real path, and cannot assert in a doctest

Metadata

struct Metadata
  size: int
  modified: int
  permissions: Permissions
  directory?: bool
end

One file's stat: size in bytes, modified in epoch nanoseconds, permissions, and directory?.

modified is a raw int and no datetime.Instant, which leaves reading a file's size free of dragging the whole calendar in behind it. datetime.Instant.from_epoch_nanos(m.modified) is the one call that converts, for the callers that want it. Nanoseconds because seconds cannot see a file replaced twice within one second, which is the case that asked for the field.

metadata!

def metadata!(path: string) -> Result<Metadata, sys.FsFailure> [fs_read]

Stat path, following symlinks. A missing path is Err(NotFound) with the path attached, and no empty success; see the retrieval rule in the module header.

One stat(2) answers all four fields. They arrive together for that reason, and never as four calls.

This follows a symlink, and metadata!(link).directory? therefore describes the target. A DirEntry from list_dir! does not, and reports the link itself, which is the difference that leaves walk! unable to diverge on a cycle. Ask this about a path you mean to open; read the entry when you are enumerating. @no-doctest: stats the filesystem; needs a real path, and cannot assert in a doctest

list_dir!

def list_dir!(path: string) -> Result<List<sys.DirEntry>, sys.FsFailure> [fs_read]

List the entries of the directory at path as sys.DirEntry values, sorted by name. A missing path or a non-directory is Err. @no-doctest: reads a real directory; needs one on disk, and cannot assert in a doctest

_WalkStep

struct _WalkStep
  rel: string
  is_dir: bool
end

One entry the walk has still to deal with, as a path relative to the walk root: a directory whose entries are not listed yet, or a file not yet reported.

walk!

def walk!(root: string) -> Result<List<string>, sys.FsFailure> [fs_read]

Every file under root, as a path relative to it ("style.css", "docs/reference/index.html"). Depth first, each directory's entries in list_dir! order (sorted by name) with a subdirectory's contents standing in for its entry; directories are not themselves reported. The order is a function of the tree alone, which is what lets a generator's output be byte-identical across runs.

Symlinks are not followed. sys.DirEntry.is_dir does not report one as a directory, and a link is therefore reported as a file and its target never descended into, which leaves the walk unable to diverge on a cycle and a link out of the tree unable to widen it.

The pending entries are an explicit worklist and no recursion, and a deep tree therefore costs heap and no stack. The first directory that cannot be listed ends the walk with its error.

Every file means every file: narrowing is the caller's, and glob is what narrows it. glob.compile("src/**.hk") once, then filter on pattern.matches?. The paths this returns are the slash-separated form a pattern matches against. @no-doctest: walks a real tree; needs one on disk, and cannot assert in a doctest

walksteps

def _walk_steps(prefix: string, entries: List<sys.DirEntry>) -> List<_WalkStep>

A directory's entries as pending steps, in listing order. The caller reverses before pushing, which pops the first entry first.

mkdir!

def mkdir!(path: string, parents: bool) -> Result<(), sys.FsFailure> [fs_write]

Create the directory at path. With parents, also create missing parent directories (mkdir -p) and treat an existing directory as success. @no-doctest: creates a directory; side-effecting, and cannot assert in a doctest

delete!

def delete!(path: string, recursive: bool) -> Result<(), sys.FsFailure> [fs_write]

Remove the file or directory at path (a symlink is removed and never its target). A non-empty directory needs recursive. @no-doctest: removes a real path; side-effecting, and cannot assert in a doctest

copy!

def copy!(src: string, dst: string, overwrite: bool) -> Result<(), sys.FsFailure> [fs]

Copy the file at src onto dst. Only regular files are copied; a directory src is Err. overwrite decides an existing dst: false is Err(AlreadyExists), true replaces it. @no-doctest: writes the filesystem; side-effecting, and cannot assert in a doctest

rename!

def rename!(src: string, dst: string, overwrite: bool) -> Result<(), sys.FsFailure> [fs_write]

Move src to dst atomically: the destination goes from old to new with nothing in between, which copy-then-delete cannot promise. overwrite decides an existing dst (false is Err(AlreadyExists)); the replacing form is what write-to-temp-then-swap needs.

Endpoints on different filesystems are Err(CrossDevice) and never an unannounced copy-and-delete: a caller that wants that fallback writes it, and then knows it is not atomic. @no-doctest: writes the filesystem; side-effecting, and cannot assert in a doctest

find_up!

def find_up!(start: string, marker: string) -> Result<Option<string>, sys.FsFailure> [fs_read]

Walk from start towards the filesystem root looking for marker, answering the first directory that has one. Ok(None) means the walk reached the root without finding it.

This is how git, cargo and npm find a project root, and it is easy to get subtly wrong in one place: the termination. The walk stops when path.dirname stops changing the directory, and never at a hard-coded "/": a relative start bottoms out at "." and never sees a slash at all, and a hard-coded root would spin there forever.

marker may name a file or a directory; the test is existence and never kind. That is what .git needs, being a directory in a clone and a file in a worktree. The walk is lexical over the path it is given, path being a pure textual module and nothing here resolving symlinks first. Pass an absolute path when the caller's working directory should not matter.

A stat failure part-way up (a permission error, say) is Err and no unannounced Ok(None): "I could not look" and "it is not there" are different answers, and only one of them means keep searching elsewhere. @no-doctest: walks the real filesystem; the answer depends on where the tree is