base64

stdlib/extra/base64.hk: RFC 4648 base64 for bytes.

The other bytes-to-text codec (hex is the simpler one): three bytes become four characters, and unlike hex this one has a grouping, a padding rule and two alphabets. Both alphabets share the first 62 characters and differ only in the last two: + and / for the standard form, - and _ for the URL-safe one (RFC 4648 §5), which is what a JWT, a query string and a file name need.

Encoding is two functions and no single one with a flag, per the house taste against option booleans: encode is standard and padded, encode_url is URL-safe and unpadded, which is the pairing each caller wants. Decoding is one permissive function, the alphabets being unable to collide (no character means one thing in one and another in the other) and a decoder that refuses an unpadded input rejects most of the web. Padding, when present, must be correct.

Pure and whole-input: every entry point is @encapsulated, and the builder it writes into is allocated and finished inside the call with nothing mutable escaping. That leaves them callable from meta and from ordinary pure code, on both tiers. There is no streaming form.

_common

_common: string = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"

The shared 62 characters, in RFC 4648 order.

Base64Error

type Base64Error
  BadCharacter(int)
  BadLength(int)
  BadPadding(int)
end

Why a base64 string could not be decoded, with the byte offset at which the problem was found (the json.JsonError idiom).

encode

def encode(input: bytes) -> string

Encode input in the standard alphabet with = padding: the form for a MIME header, an HTTP basic-auth credential, or anything RFC 4648 §4 calls base64 without qualification.

encode("".to_bytes())    => ""
encode("f".to_bytes())   => "Zg=="
encode("fo".to_bytes())  => "Zm8="
encode("foo".to_bytes()) => "Zm9v"

encode_url

def encode_url(input: bytes) -> string

Encode input in the URL-safe alphabet with no padding: the form a JWT, a query parameter and a file name want, since +, / and = all need escaping there. decode reads it back without being told which it is.

encode_url("".to_bytes())    => ""
encode_url("f".to_bytes())   => "Zg"
encode_url("fo".to_bytes())  => "Zm8"
encode_url("foo".to_bytes()) => "Zm9v"

decode

def decode(text: string) -> Result<bytes, Base64Error>

Decode base64 text, accepting either alphabet and either padding. Padding is optional, but a = that appears must end the input and leave a group of two or three characters; a group of one is BadLength, and any character outside both alphabets is BadCharacter at its offset. Nothing is skipped, neither whitespace nor a newline: a codec that drops characters unannounced cannot tell a wrapped MIME body from a corrupted one.

decode("Zm9v").unwrap_or("".to_bytes()).length      => 3
decode("Zg==").unwrap_or("".to_bytes()).length      => 1
decode("Zg").unwrap_or("".to_bytes()).length        => 1
decode("Z").unwrap_or("rejected".to_bytes()).length => 8

_encode

def _encode(input: bytes, extra: string, pad: bool) -> string

The four characters of one 24-bit group, or fewer at the end. extra names the last two alphabet characters and pad whether to fill the group out.

_at

def _at(input: bytes, i: int) -> u32

Byte i of input widened, or 0 past the end: the zero fill RFC 4648 specifies for a short final group. u32 throughout the group arithmetic: three bytes and four 6-bit fields both fit, and the narrowing back to u8 is the mask, which leaves nothing needing a range check and nothing able to crash.

_char

def _char(v: u32, extra: string) -> string

The alphabet character for the low 6 bits of v.

_value

def _value(ch: u8) -> Option<u32>

The 6-bit value of one character in either alphabet, or None. Arithmetic and no scan, which spares either alphabet a lookup.

datalength

def _data_length(raw: bytes) -> Result<int, Base64Error>

How much of raw is data and not padding, or why the padding is wrong. At most two trailing =, nothing after them, the data left over must not end a group with a single character, and any padding present must be the amount RFC 4648 writes for that group - "optional, but correct when present" (HANKI.md section 17), which is what the caller is told.

padsfor

def _pads_for(rest: int) -> int

How many = RFC 4648 writes after a data run of this length. A residue of 1 never reaches here: no byte count produces a group of one character, and _data_length has already rejected it as BadLength.

decodedata

def _decode_data(raw: bytes, stop: int) -> Result<bytes, Base64Error>

The bytes of raw's first stop characters, four at a time. Every group yields one byte per whole 8 bits in it: 4 characters give 3, 3 give 2, 2 give 1.

_offset

def _offset(e: Base64Error) -> int