> ## Documentation Index > Fetch the complete documentation index at: https://mehen.ophi.dev/llms.txt > Use this file to discover all available pages before exploring further. # Add a new language > How to add a new language analyzer to mehen — pick a parser, implement the LanguageAnalyzer trait, and register the analyzer. Each language is owned by a single per-language analyzer crate at `crates/mehen-/`. The analyzer: * Pins its own parser (a language-specific parser like Ruff / Oxc / Mago / Prism / `ra_ap_syntax`, or a tree-sitter grammar). * Owns metric interpretation for that language's syntax — what counts as a decision, an operator, a method, a comment, etc. * Returns `LanguageAnalysis` (owned, `Send + 'static`) so `mehen-engine` can analyze files in parallel and never holds onto parser arenas. For tree-sitter-backed languages the analyzer also owns its `grammar.rs` kind enum, generated from the pinned grammar's `node-kind` table. ## Choosing a parser The first decision is the parser. mehen prefers a language-specific parser when one exists with mature Rust bindings and rich AST/semantic coverage; tree-sitter is the default fallback when no such parser is available. | Parser style | Examples in mehen | When to pick | | ----------------- | ------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Language-specific | Ruff (Python), Oxc (TS/JS), Mago (PHP), Prism (Ruby), `ra_ap_syntax` (Rust), pulldown-cmark (Markdown) | A maintained Rust crate with a typed AST, semantic model, or rich grammar coverage exists. | | ANTLR | Kotlin (official Kotlin spec grammar), Java (grammars-v4) | A high-quality, maintained ANTLR v4 grammar exists but no Rust-native parser does. Generated via [`ophi-dev/antlr-rust-runtime`](https://github.com/ophi-dev/antlr-rust-runtime); the CST is far more semantically named than tree-sitter's. | | Tree-sitter | Go, C, PowerShell | No mature language-specific or ANTLR parser is available, or tree-sitter's grammar quality is the best fit. | ## Adding a language-specific parser Add the parser crate(s) to `crates/mehen-/Cargo.toml`. Pin an exact version (or a git revision tagged for the release) so mehen's behavior is reproducible. In `crates/mehen-/src/lib.rs`: * Define `Analyzer` and implement `mehen_core::LanguageAnalyzer` for it. * Walk the parser's typed AST and emit metrics through `mehen_metrics::{State, MetricTreeBuilder, …}` and the per-metric helpers. * Make sure the parser's arena, source buffer, and parser state do not escape the `analyze` call — `LanguageAnalysis` must be `Send + 'static`. Add it to `mehen-engine`'s registry (`crates/mehen-engine/src/registry.rs`) so `Language::` dispatches to it. Per-metric integration tests under `crates/mehen-/tests/` — typically one file per metric family, snapshotting the rendered metric JSON via `insta`. ## Adding an ANTLR-backed language ANTLR is a first-class backend (`AnalysisBackend::Antlr`), used when a high-quality ANTLR v4 grammar exists but no Rust-native parser does. The shared crate `mehen-antlr` provides the runtime re-export, token-span conversion, recovered-error diagnostics, and hidden-channel comment (CLOC) extraction. The Rust lexer/parser modules are generated **offline** from a vendored `.g4` grammar and checked in — a normal `cargo build` never needs Java or the ANTLR jar. Copy the ANTLR `.g4` files (lexer, parser, and any `import`ed grammars like `UnicodeClasses`) into `crates/mehen-/grammar/`. Record the upstream source, commit, and toolchain versions in a `PROVENANCE.md` so regeneration is reproducible (see `crates/mehen-kotlin/grammar/`). Add an `AntlrTarget` to `xtask/src/antlr.rs::TARGETS` (slug, crate dir, grammar dir, lexer/parser filenames). ```bash theme={null} export MEHEN_ANTLR_JAR=/path/to/antlr-4.13.2-complete.jar cargo install antlr-rust-runtime # provides antlr4-rust-gen (or set MEHEN_ANTLR_RUST_GEN) cargo xtask antlr generate ``` The result lands in `crates/mehen-/src/generated/`. The generated files carry their own lint and `rustfmt::skip` attributes, so the owning analyzer can include them with plain `pub(crate) mod ...;` declarations unless another crate needs direct access. Never edit the generated modules. `cargo xtask antlr check-generated` guards drift in CI when the toolchain is available (it skips, exit 0, where the tools are absent). Depend on `mehen-antlr` (the shared helpers) **and** `antlr4_runtime` (the generated modules reference the runtime by its real crate name). Both reach the same workspace-pinned version, so the pin stays in one place. Parse via the generated `Lexer`/`Parser`, then walk the `ParseTree` with your own recursive `Visitor` (like `mehen-rust`/`mehen-ruby`). ANTLR rule contexts have **no parent pointer**, so thread any parent-dependent context (e.g. else-if detection) top-down. Match `rule_index()` against the generated `RULE_*` constants and terminal `token_type()` against the token constants. Comments are on a hidden channel (absent from the tree) — recover CLOC from the token stream via `mehen_antlr::comment_rows`. Same as the other backends: register in `crates/mehen-engine/src/registry.rs`. Per-metric integration tests under `crates/mehen-/tests/`. ## Adding a tree-sitter-backed language **Prerequisite:** a `tree-sitter-` crate compatible with the `tree-sitter` version pinned in the workspace (`Cargo.toml` `[workspace.dependencies]`). Two files must stay in sync — both reference the grammar at compile time: * `xtask/Cargo.toml` — the kind-enum generator imports the grammar at codegen time. * `crates/mehen-/Cargo.toml` — the analyzer imports the grammar at runtime to drive `tree_sitter::Parser`. Workspace-managed grammars (those listed in root `Cargo.toml` `[workspace.dependencies]`) can be referenced as `{ workspace = true }` from both places. Inline-pinned grammars must be kept in lockstep manually. Add a `GeneratorTarget` to `xtask/src/tree_sitter.rs::TARGETS`: ```rust theme={null} GeneratorTarget { slug: "go", enum_name: "Go", crate_dir: "crates/mehen-go/src", language: || tree_sitter_go::LANGUAGE.into(), }, ``` ```bash theme={null} cargo xtask tree-sitter generate go ``` The result lands at `crates/mehen-go/src/grammar.rs`. Never edit `grammar.rs` directly. `cargo xtask tree-sitter check-generated` runs in CI and fails the build if the checked-in file doesn't match a fresh render. Same pattern as a language-specific parser, but use the generated `crate::grammar::` enum for kind-id matching — it deduplicates positional kinds and exposes mnemonic identifiers (`PLUS`, `EQ_EQ`, etc.). Same step as above: register in `crates/mehen-engine/src/registry.rs`. Per-metric integration tests under `crates/mehen-/tests/`. ## Bumping a pinned grammar When dependabot bumps a `tree-sitter-` version (or you do it manually): 1. Update both `xtask/Cargo.toml` and `crates/mehen-/Cargo.toml` to the new version. The `regenerate-grammars` workflow does this automatically for inline-pinned grammars. 2. Run `cargo xtask tree-sitter generate --all` and commit the regenerated `grammar.rs` files. 3. CI's `cargo xtask tree-sitter check-generated` will fail until the regenerated files are committed. ## Validation ```bash theme={null} cargo check --workspace cargo nextest run --all-features cargo insta test --workspace --all-features --check --unreferenced reject \ --test-runner nextest --no-test-runner-fallback --disable-nextest-doctest cargo clippy --all-targets --all-features --locked cargo xtask tree-sitter check-generated ``` ## See also * [Update grammars](/developers/update-grammars) — bumping pinned tree-sitter versions. * [Implement LoC](/developers/loc) — example of a metric trait implementation.