# Changelog All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] ## [0.7.0] - 2026-10-21 ### Fixed - **A failed PEG parse says where it went wrong**, by both backends and in `run_sequence`: at the furthest token any alternative reached, with what was found there and what would have been accepted -- `unexpected "}" -- expected one of: "[", "]", INT`, or `unexpected end of input -- ...` -- in `found` and `expected`. It used to say only that the root did not match, at the first token. What a lookahead failed to match is not counted, and input left over after a complete match is reported at its first token rather than at the trivia before it. Every engine now agrees: LR and GLR already stopped at the failing token. ### Changed - **The README warns, at the top, that this library is under heavy development** and not for production use before `1.0.0`: things can and will break, and the specification can and will change, between any two releases until then. - **The warning reaches hex.pm and HexDocs too**: the package description starts with it, so search results show it, and HexDocs shows the README's warning as a warning box -- GitHub's `> [!WARNING]` alert, which ExDoc does not render, is rewritten into ExDoc's own admonition when the docs are built (the `docs` alias; the README is unchanged). ## [0.6.0] - 2026-10-20 ### Fixed - **A capture owns the reference it directly wraps**, by every engine: in `sign:SIGN?`, `items:item*` or `n:X{1,3}` the reference is captured under the capture's name only. The PEG backends also captured it under its own name, so `sign:sign?` came out as a list of two -- the rule's node and the text -- and `sign:SIGN?` handed an action both `sign` and `SIGN`. The lossless tree still has a node for the rule. - **Every engine keeps the captures inside a captured group**, as the PEG backends always did: `all:(k:A B)` gives `all`, `k` and `B`. LR and GLR dropped them, and reported a repeated reference inside one (`many:A*`) as an empty `A: []`. - **The Aether guides**: `Grammar.VM` runs only `@engine peg` (`Grammar.LR` and `Grammar.GLR` run the others, interpreted), and `TRIVIA?` at the ends of a grammar is no longer needed since every engine trims the skip tokens around the input. - **An action's error says where**, by every engine and both backends, lossless or not, and in `run_sequence`: an error a `handle_rule/3` or `handle_token/3` returns with no line is placed where that rule or token matched, and shown with its line and a caret. Every parser records where each capture matched; it used to keep only the text, so the runtime had nothing to place such an error with. ### Added - **Captures and default nodes have spans**: an action reads where a capture matched as `capture.span`, `{line, column, length}`, and a default `%Ichor.Node{}` has its rule's. ### Changed - **Needs `ichor_runtime` 0.5**, for the recorded positions (`Ichor.Capture.at/3`, `Grammar.Source.Locator`). A PEG parser generated by an earlier `ichor` still runs on it, without positions; regenerate an LR or GLR one, whose `parse/2` would otherwise hand back the recorded positions with its captures. - **`parse/2` of a generated module returns the captures as before**, without the recorded positions; a `@native(...)` rule's or lexeme's `rule_matchers` hand it plain nodes, as they always have. ## [0.5.1] - 2026-10-19 ### Added - **`mix ichor.conformance` answers `lossless`**: the lossless tree of a text, as the Ichor Conformance Suite writes it -- a node as `{:rule, name, offset, size, line, column, children}`, a token as `{:token, name, text, trivia, offset, line, column}`, an inline literal's token named nil -- through either backend, so every implementation is held to the same tree, offsets included. ## [0.5.0] - 2026-10-19 ### Added - **The lossless mode** (`design/lossless-mode.md`): a parser generated with `lossless: true` (`use Ichor`, `Ichor.generate/4`, `Ichor.generate_from_grammar/3`) or `mix ichor.gen --lossless` gains `parse_lossless/1,2` and `run_lossless/1,2`. They return the lossless tree of an input (`Ichor.Syntax`, in `ichor_runtime`): every byte, trivia and punctuation included, under every grammar rule that matched, with byte offsets, lines and columns; `run_lossless` returns the ordinary value beside it. For every engine and both backends -- `Grammar.VM`, `Grammar.LR` and `Grammar.GLR` have `parse_lossless/3` and `run_lossless/4` -- and the same tree from each. Without the option, the generated code is what it was; with it, only the module grows. ### Fixed - **An LR or GLR rule repeated three times or more is a flat list** (through `ichor_runtime` 0.4.0): `doc := entry+` with three entries gave `[a, [b, c]]`, which `Ichor.Actions` could not evaluate. ## [0.4.0] - 2026-10-18 ### Added - **`mix ichor.conformance interpreted|compiled REQUEST ANSWER`**, this implementation's adapter for the Ichor Conformance Suite (`Ichor.Conformance`). It reads the suite's JSON request, makes a parser from each case's grammar through the named backend -- `Grammar.VM`/`Grammar.LR`/`Grammar.GLR`, or a module generated by `Ichor.generate_from_grammar/2` -- and answers every case with its default tree, or its error's stage, line and column, as the suite's typed JSON. The suite holds each backend to its cases separately, which is what shows the two agree. A case's grammar may be written in any notation Ichor reads -- Aether, ABNF, BNF, ISO EBNF, PEG -- told by its extension or an `@style` line. - **`Grammar.VM.tokenize/3`**: a grammar's token stream for an input, interpreted, for every engine -- what a generated module's own `tokenize/2` answers. `Grammar.LR` and `Grammar.GLR` lex through it. - **`mix ichor.conformance` answers the toolkit cases**: `"pratt"`, `"unify"`, `"solve"` and `"walk"` run `Ichor.Toolkit.Pratt`, `Ichor.Backtrack.Bindings`, `Ichor.Backtrack.Tree` and `Ichor.Toolkit.TermWalk` on data a case gives, through one standard set of callbacks and one term module (`Ichor.Conformance.Toolkits`). - **`mix ichor.conformance` answers `"render"`**: the first error exactly as `Ichor.Error.format/1` writes it, or the tree when the text parses. - **`mix ichor.conformance` answers `"tokens"`** as well as `"parse"`: the token stream, trivia included, as `{name, text, line, column}`, an inline literal's generated token named nil. - **W3C EBNF can be imported** (`:ebnf_w3c`), as the other four notations always could: `Ichor.EBNF.W3C` read it but `Ichor.GrammarImport` did not offer it, which the Praxis port already called an oversight. A file says which EBNF it is by `.w3c.ebnf` or `.iso.ebnf`, or `@style ebnf-w3c` / `@style ebnf-iso`; a plain `.ebnf` is ISO, as before. - **`Ichor.GrammarImport.styled/2`**: which notation a grammar file is written in, by its `@style` line or its extension, and the source to hand that notation's reader. `mix ichor.gen` and the conformance adapter both use it, so they never disagree about what a file is. ### Changed - Licensed under the Apache License 2.0 (`Apache-2.0`); it was MIT. - **A rule that captures nothing is the text it matched.** Its body is captured as `:text` -- the input from its first token to its last -- so its default value is that string, and an Actions module finds it in `%{text: ...}`. It used to be an empty node, and what it matched was lost to whatever the parse was fed to; in an imported ABNF, BNF, EBNF or PEG grammar, which have no capture syntax, every terminal was. Decided by the grammar, never by the input (`Grammar.Analysis`). A generated parser changes when it is regenerated: its capture-less rules come out as text, where they were empty nodes. ### Fixed - **Trivia around the input is skipped by every engine and both backends.** `run/2` refused a leading `@skip` token that `run_sequence/2` skipped, so `" 2"` parsed through one and not the other, and an `@engine lr` grammar refused trailing trivia that PEG and GLR accepted. The stream is now trimmed of `@skip` tokens before its first other token and after its last (`Grammar.Source.trimmed/2`); between two tokens they are the grammar's, as before. - **Every parse error says where.** A root that does not match is placed at the token it was tried at, and "unexpected end of input" at the end of the input -- line 1, column 1 for no input at all. Both used to carry no line or column. - Both found by the Ichor Conformance Suite. Needs `ichor_runtime` 0.3. ## [0.3.1] - 2026-10-07 ### Added - **`guides/aether/AETHER_GRAMMAR.md`: Aether's own syntax, written as an Aether grammar.** `Aether.Reader` stays hand-written; the guide is a description, and `test/aether/grammar_doc_test.exs` keeps it honest by compiling the guide's grammar and checking it accepts exactly what `Aether.Reader` accepts, across every `.aether` file in the repository. - **Guides for the two Mix tasks**, `guides/mix_tasks/ICHOR_GEN.md` and `ICHOR_TOKENS.md`, in a "Mix Tasks" group of their own in the docs. - **The package links its GitHub Pages documentation** beside the repository. ### Fixed - **A `name:` capture inside a token body is now a reader error** at the capture's name: "captures are only valid in rule bodies". It used to be accepted, after which `Grammar.VM` crashed with a `FunctionClauseError` on the first match. Captures were always documented as rule-only. - **References to types in the docs resolve.** They were written as if they named functions, and ExDoc warned that no such function exists; they carry the `t:` prefix now. ### Changed - **`mix.lock` is no longer checked in**, as befits a library: it only ever pinned what this repository's own CI resolved, never what a user gets. CI runs Elixir 1.19.5. ## [0.3.0] - 2026-08-03 ### Changed - **Breaking:** bumped `ichor_runtime` to `~> 0.2` (from `~> 0.1.0`), whose 0.2.0 changed raw capture data (`t:Ichor.Capture.node_t/0`'s `:rule` variant, and therefore every `parse/1`-style bare-recognizer function `use Ichor`/`Grammar.Native`/`Grammar.Native.LR`/`Grammar.Native.GLR`/ `Grammar.LR`/`Grammar.GLR`/`Grammar.VM` generate or expose) from a plain `%{name => value}` map to an ordered `[{name, value}]` list (`t:Ichor.Capture.raw_captures/0`), fixing sibling-capture evaluation order depending on a plain map's own (cross-OTP-version-unstable) iteration order instead of true first-occurrence source order. Ported the same fix to `Grammar.VM.TokenInterpreter` (this project's own reference/interpreted PEG path, which ichor_runtime doesn't own) for parity with the native backend, and updated `Grammar.Native.RuleCompiler`'s generated code and `Ichor.GrammarImport`'s hand-written capture pattern-matches to the new list shape. Any hand-written `Ichor.Actions` module that pattern-matches or constructs `raw_captures`/`node_t()` values directly (e.g. for macro-expansion via `Ichor.Actions.evaluate_node/3`) needs the same `%{name: value}` -> `[{name, value}]`/`[name: value]` update -- the already-*evaluated* `t:Ichor.Actions.captures/0` map handed to `handle_rule/3` callbacks is unchanged. ## [0.2.1] - 2026-07-29 ### Changed - **Documentation reframed around three ways to run a grammar**, with `mix ichor.gen` established as the recommended default for anything shipping to production: README gained a "Three ways to run a grammar" section (`mix ichor.gen` / `use Ichor` / the raw `Aether.Parser` + `Grammar.Analysis` + `Grammar.VM` pipeline) spelling out the concrete reason for that recommendation -- `use Ichor` needs the real `Ichor` module present at compile time in whatever environment you compile in, so `ichor` can't be `only: :dev, runtime: false` in any app that still has one, including a `mix release` build (which normally compiles under `MIX_ENV=prod`); `mix ichor.gen` output has no macro dependency on `ichor` at all, so it's the only path that actually allows the dev-only split `ichor_runtime` exists to enable. The tutorial gained a new §8, "Loading a grammar at runtime", with a worked plugin-loader example for the third path (previously only mentioned in passing, never actually demonstrated); the cheatsheet gained a "Which one do I want?" comparison table and was reordered to match; the `Ichor`/`Mix.Tasks.Ichor.Gen`/`Grammar.VM` moduledocs now name and cross-reference all three paths for readers landing there directly via ExDoc/hex search. ### Fixed - Three pre-existing broken anchor links across the guides (`guides/ TUTORIAL.md`, `guides/CHEATSHEET.md`, `guides/aether/TUTORIAL.md`, `guides/EXAMPLES.md`) -- ExDoc's markdown renderer turns a period inside a backtick-quoted heading (`` `mix ichor.gen` ``, `` `Ichor.Actions` ``) into an extra hyphen when generating the heading's own anchor id (`mix-ichor-gen`, `ichor-actions`), not just stripping it, which none of the existing cross-reference links accounted for. ExDoc doesn't validate these at build time, so they'd been silently dead since introduced. ## [0.2.0] - 2026-07-29 ### Added - **`mix ichor.gen`**: runs the parse/analyze/native-codegen pipeline ahead of time and writes the result to a plain `.ex` file, instead of `use Ichor` re-running it inside a macro expansion on every compile. Every generated helper function is emitted as `def` rather than `defp` (macro-originated code is exempt from Elixir's unused-function check; written out as ordinary source, it isn't). `Ichor.generate/3` is the new public entry point both `__using__/1` and the task share. - **`Ichor.GrammarImport`**: turns an ABNF/BNF/ISO-EBNF/PEG ruleset (as `Ichor.ABNF`/`Ichor.BNF`/`Ichor.EBNF.ISO`/`Ichor.PEG` produce them -- not a runnable `Aether.Grammar` on their own) into one that is: picks a root (the source's own first-declared rule, overridable), always `@noskip`, auto-promotes bare literals/char-classes into synthetic tokens, and -- for ABNF specifically -- fills in whichever of RFC 5234 Appendix B's "core rules" (`ALPHA`, `DIGIT`, `CRLF`, ...) the source references but never defines, restricted to their transitive closure so unreferenced, overlapping core rules (`CHAR`/`OCTET` overlapping `DIGIT`, e.g.) can't win a maximal-munch tie against a token a rule actually meant. `mix ichor.gen` now accepts any of these formats directly (`.abnf`/`.bnf`/`.ebnf`/`.peg`, auto-detected from the file extension, or forced with an `@style` pragma line), with a new `--root NAME` flag for the non-Aether entry-rule override. - **[`ichor_runtime`](https://github.com/joetjen/ichor_runtime)**: split out of `ichor` proper -- the ~24 modules a generated parser (from `mix ichor.gen` or `use Ichor`) actually calls at runtime (`Ichor.Actions`, `Ichor.Error`, the compiled Tokenizer/Parser combinators, the LR/GLR shift-reduce/GSS runtime, and standalone `Ichor.Toolkit.Pratt`/`TermWalk`/`Ichor.Backtrack`), with everything dev-time-only (the Aether front-end, format importers, `Grammar.Analysis`, the LR/GLR table builder, both codegen backends) staying in `ichor`. `Grammar.LRTable` was split in the process: the struct plus `current_terminal/3`/`unexpected_error/2` (genuinely called at match time) moved to `ichor_runtime`; table construction (`build/1`/`conflicts/1`, which need `Automaton`/`Desugar`/`Sets`) stays in `ichor` as the new `Grammar.LRTable.Builder`. Independently published ([`ichor_runtime` 0.1.0](https://hex.pm/packages/ichor_runtime)) and maintained -- its own repository, its own release cadence -- not a subdirectory of this one; `ichor` depends on it via a normal `~> 0.1.0` Hex version constraint (its own interpreted backends need it too). ### Fixed - `evaluate_node/3` moved from the top-level `Ichor` module to `Ichor.Actions.evaluate_node/3` -- found while migrating a real downstream consumer (`dextrin`) to the `ichor`/`ichor_runtime` split: it's a genuine runtime entry point (any grammar with macro-like features needs it to evaluate its own expansion, not just LISP's worked example), not a compile-time-only concern like `generate/3`/ `__using__/1`, which are the only two things that actually belong in the dev-only `ichor` package. Left in `Ichor` proper, it would have been unavailable at runtime to anything depending on `ichor_runtime` alone, exactly as the split intends. - `Ichor.Toolkit.Pratt`, `Ichor.Toolkit.TermWalk`, and `Ichor.Backtrack` (+ `.Bindings`/`.Term`/`.Tree`) moved from `ichor` to `ichor_runtime` -- the same class of gap as `evaluate_node/3` above, found migrating another real downstream consumer (Aletheia): a `@native(...)` rule's own callback module (Prolog-style operator-precedence parsing, the worked example this whole feature is motivated by -- see `test/prolog/`) calls `Ichor.Toolkit.Pratt.parse/4` on every single parse, not just once at grammar-generation time, and a consuming engine built on `Ichor.Backtrack` calls into `Bindings`/`Tree` on every resolution step. None of the three have any dev-only dependency of their own (`Pratt` operates on a plain map, `Backtrack` is self-contained, `TermWalk` only needs `Backtrack.Term`), so keeping them in the dev-only package was an oversight in the original split, not a deliberate exclusion. `Ichor.Toolkit.TypeScheme` (which stays in `ichor` -- it's a compile-time type-checking concern) depends on both `Bindings` and `TermWalk`; unaffected by the move since `ichor` already depends on `ichor_runtime`. ## [0.1.1] - 2026-07-28 ### Added - `@native(...)`/`@hint(...)` at **rule position** (`Grammar.IR.Custom`): an escape hatch for grammar rules plain PEG can't express — most notably a runtime-mutable operator-precedence table (Prolog's `op/3`, Haskell fixity declarations, Coq/Agda/Lean mixfix notation), dispatching to an author-supplied `c:Ichor.CustomRule.match/4`-shaped callback with access to the token stream, the current position, the grammar's own `context` (read-only, threaded via `run_sequence/4` across top-level forms), and `rule_matchers` restricted to the node's own declared dependencies. - `@native(...)`/`@hint(...)` at **token position** (`Grammar.IR.CustomLexeme`/`Ichor.CustomLexeme`): the analogous escape hatch for lexing that fixed maximal-munch tokenization can't express — heredocs (the terminator is read off the source itself), string interpolation (a token needing to recurse into a rule mid-scan), and in principle TeX-style `\catcode` mutation. Includes a new "re-lex and match rule R starting at a given character position" primitive (`rule_matchers.name.(input, char_pos)`), supported by both backends. - **`@engine peg | lr | glr`** pragma (default `peg`, fully backward compatible): selects the parsing algorithm for `@root`, independent of execution strategy (interpreted vs. compiled): - **`Grammar.LR`** (and `Grammar.Native.LR` for compiled codegen): a from-scratch SLR(1) table (`Grammar.LRTable`) plus a deterministic shift-reduce engine, for grammars an author has already verified are conflict-free — zero GSS/backtracking bookkeeping, and (compiled) fully inlined into one Elixir function per automaton state. - **`Grammar.GLR`** (and `Grammar.Native.GLR`): genuine Tomita-style GLR over the same SLR(1) table, forking at every real conflict via a graph-structured stack (`Grammar.GLR.GSS`) that shares/merges history instead of naively duplicating parallel stacks, resolving multiple valid derivations via a declared-order tie-break (`Grammar.LRTable.Automaton`'s own canonical "shift before reduce, reduces by production id" conflict-cell ordering) — this is what lets it parse inputs plain PEG's greedy, non-backtracking commit can never recover from, and pick a deterministic winner for genuine CFG-level ambiguity a PEG grammar could never even express. - `Grammar.Analysis` skips its PEG-specific left-recursion rewrite for non-`peg` grammars (left recursion is what bottom-up shift-reduce parsing wants, not a hazard); both backends guard against running a mismatched-`@engine` grammar through the wrong entry point. - **`@keywords` / `@refine`**: token-stream reclassification after tokenization, before parsing (`Grammar.Lexer.reclassify/2`) — `@keywords WORD { "if" -> KEYWORD_IF, ... }` is table-lookup sugar for the general case, `@refine(Module, :function)` dispatches to an author-supplied callback with the already-reclassified stream so far (enabling lookbehind-dependent reclassification, e.g. JavaScript's regex-literal-vs-division-operator ambiguity). - **`Ichor.Backtrack`**: a generic lazy-search-plus-unification substrate for logic-language evaluation models (SLD-resolution and friends) — `Ichor.Backtrack.Tree` (`unit`/`fail`/`disjunction`/ `conjunction`/`once`, pulled lazily via `next/1`, depth-first/ left-to-right like Prolog's own clause-order backtracking), `Ichor.Backtrack.Bindings` (a substitution: `bind`/`resolve`/`unify`/ `unify_occurs_check`, over any `Ichor.Backtrack.Term` implementation an author supplies for their own term representation). - **`Ichor.Toolkit`**, small IR-agnostic building blocks extracted from patterns Ichor's own compiler internals had already independently hand-rolled more than once — useful to a grammar author's own downstream semantic analysis/type inference/codegen, never run by Ichor itself: - `Fixpoint`/`Graph` — "iterate a step function to convergence," and reachability/cycle detection over an arbitrary graph. - `Scope` — a generic nested lexical scope/symbol table. - `TypeScheme` — Hindley-Milner-style let-polymorphism (`generalize`/`instantiate`/`resolve_deep`) built on `Ichor.Backtrack.Bindings.unify_occurs_check/4`. - `Codegen` — small `quote`/`unquote` mechanics (`fresh`/`vars`/ `indexed_vars`/`capture`/`clause`) for authors hand-writing their own Elixir-codegen backend. - `Result` — `reduce_ok`/`map_ok`, an `Enum.reduce`-with-early-exit- on-failure pair for the "walk a collection, thread an accumulator through a fallible step" idiom. - `Pratt` — precedence-climbing (Pratt) expression parsing over a runtime-mutable operator table (prefix/infix/postfix, including the genuinely ambiguous case of an operator registered as both infix and postfix at once). - `Layout` — the Python/Haskell/F#-style off-side-rule (`INDENT`/ `DEDENT` token synthesis) algorithm, an alternative strategy to Aether's own `@indent`/`@samecol`. - `TermWalk` — generic `fold`/`rewrite` recursion over any `Ichor.Backtrack.Term` implementation. - Native GLR/LR codegen (`Grammar.Native.LR`/`Grammar.Native.GLR`): the SLR(1) table is built once, at Elixir-compile-time, instead of on every call; `Grammar.Native.LR` fully inlines shift/reduce dispatch into one Elixir function per automaton state (a genuine Bison-style compiled parser), `Grammar.Native.GLR` compiles the action/goto lookup tables while sharing one runtime GSS engine (`Grammar.GLR.Runtime`) with the interpreted backend. - Five more worked-example grammars, chosen specifically to close gaps where a feature had only ever been exercised by a synthetic/textbook fixture, never anything relating to a real programming language, file format, or protocol: - **Prolog** (`test/prolog/`) — real clause/rule/directive syntax (facts, rules with conjunction, `:- op(Prec, Type, Name).` directives genuinely mutating the operator table mid-file via `Ichor.Toolkit.Pratt` + `run_sequence/4`'s context threading), closing the loop with `Ichor.Backtrack`: a parsed fact unifies directly against a hand-built query term through the same substrate. - **The classic dangling-else ambiguity** (`test/dangling_else/`) — real, present in C/Java/JavaScript/PHP, resolved correctly (else binds to the nearest `if`) with no special-casing needed. - **CommonMark thematic breaks** (`---` -> `
`, added to the existing Markdown grammar) — the first real use of `&expr` (`Grammar.IR.AndPred`) anywhere in the test suite. - **A classic-BNF LISP fixture** (`test/lisp/lisp.bnf`) — `Ichor.BNF`'s first real-world-named fixture. - **A W3C-EBNF XML subset** (`test/xml/`) — `Ichor.EBNF.W3C`'s first real-world-named fixture, keeping the real XML 1.0 spec's own production names. ### Changed - Split `Aether.Parser` into `Aether.Reader` (pure-syntax parsing of `.aether` source into a concrete syntax tree, no desugaring) and `Aether.Eval` (that CST into `Grammar.IR`/`Aether.Grammar`, owning predefined-token override/use tracking, case-insensitivity resolution, character-class/regex desugaring, inline-literal promotion, and `@skip` splicing). Mirrors the Reader/Eval split generated grammar code already has (`parse/1` vs. `run/1,2`) and the ABNF/BNF/EBNF/PEG importers already inherit from it. `Aether.Parser.parse/2` remains the combined entry point; observable behavior is unchanged. - Split the tokenizer/lexer/parser pipeline into four explicit stages, shared identically by both backends: `Grammar.Source` (UTF-8 validation), `Grammar.VM.Tokenizer`/`Grammar.Native.Runtime.Tokenizer` (renamed from `Lexer`/`Grammar.Native.Runtime` — maximal-munch char-level matching), `Grammar.Lexer` (new — `@keywords`/`@refine` reclassification, shared by both backends since it's pure post-tokenization data processing), and the existing token-level Parser stage (`Grammar.Native.Runtime.Parser`, split out of `Grammar.Native.Runtime`). - `Aether.Grammar` gained `rule_order` (mirroring the existing `token_order`) and an `engine` field (`:peg` by default). - Renamed `LICENSE.txt` to `LICENSE`. ## [0.1.0] - 2026-07-26 ### Added - **Aether**, Ichor's own grammar language: `ALL_CAPS` tokens matched by maximal munch, `snake-case`/`kebab-case` rules matched by ordered PEG choice, string/character-class/regex-literal primitives, quantifiers (`*`/`+`/`?`/`{n,m}`), lookahead predicates (`&`/`!`), named captures, automatic `@skip` whitespace-splicing (with `@noskip` and `~` to opt out), `@case_insensitive`, POSIX bracket classes, and `@indent`/ `@samecol` for layout-sensitive grammars. - `Grammar.IR`, the normalized AST every front-end compiles to and every backend consumes. - `Grammar.Analysis`: reference checks, a direct-left-recursion rewrite, an empty-repetition hazard check, and a duplicate-alternative lint. - `Grammar.VM`, an interpreted bytecode backend (LPeg-style compiled Lexer + Parser). - `Grammar.Native`, a compile-time codegen backend (`use Ichor`), generating direct Elixir function calls instead of bytecode. - `Ichor.Actions`, the behaviour connecting a parsed grammar to real evaluation, transpilation, or execution: `handle_rule/3`, `handle_token/3`, optional `finalize/1`, and a default fallback for anything a grammar's own Actions module doesn't implement. - Format importers reading ABNF (RFC 5234 + RFC 7405), classical BNF (ALGOL 60 Report convention), ISO/IEC 14977 EBNF, W3C-style EBNF (the notation the XML 1.0 spec's own section 6 uses), and PEG (Ford's paper / pest / PEG.js convention) — each producing the same `Grammar.IR`-based ruleset shape every other front-end produces. - `mix ichor.tokens`, listing every token a grammar declares in maximal-munch tie-break order. - Nine worked-example grammars exercising the library end to end: a calculator, a LISP dialect, YAML, LogQL, SQL, HTTP, regex, Forth, and a Markdown-to-HTML transpiler. - Cross-format validation: six of those worked examples (calculator, LISP, LogQL, SQL, HTTP, regex) independently re-expressed in ABNF, ISO EBNF, and PEG, verified to produce equivalent tokens and identical accept/reject behavior to their native Aether counterparts.