Performance

Shire is designed to index large monorepos quickly and answer queries instantly. This page documents benchmark methodology, results, and how to reproduce them.

Test repos

Benchmarks run against three real-world open-source monorepos covering a range of sizes and ecosystems:

Build performance

Full rebuild (no incremental cache), median of 4 iterations after a warmup run:

Build time scales roughly linearly with symbol count. The pipeline is parallelized with rayon across packages and files, with batched multi-row SQLite inserts within explicit transactions.

Incremental builds are significantly faster – only packages with changed source files are re-extracted, and an mtime pre-check skips SHA-256 computation entirely for untouched packages.

2026-09-07: The table above predates two behavior changes not yet re-benchmarked end to end. (1) Incremental builds now always scan the working tree directly instead of relying on .git/index, so unstaged edits are indexed on the next build — this adds a fixed per-build walk cost. Measured on turborepo: a no-op incremental build went from 33ms to 173ms, a one-file incremental build from 298ms to 170ms, and a full rebuild from 1,769ms to 1,558ms. (2) search_symbols now matches by prefix and against identifier sub-tokens (e.g. verify jwt matches verifyJwtToken) rather than exact substring, which changes result sets but was not separately benchmarked for latency.

Query performance

Median latency over 100 iterations per query:

All queries use SQLite FTS5 full-text search with the unicode61 tokenizer. packages_fts and docs_fts carry a prefix='2,3' index; symbols_fts and files_fts deliberately do not (a prefix query there walks a term range instead). Query latency depends primarily on result set size, not total index size.

Reproducing benchmarks

Shire includes an autoresearch binary for reproducible benchmarking. It is a development tool, not part of the shipped CLI: it lives behind the non-default bench cargo feature, so every invocation needs --features bench.

The lifecycle and quality phases rewrite source files in the target repo. They refuse to run against a repo with uncommitted changes to tracked files (git status --porcelain --untracked-files=no must be empty, and an indeterminable state — no git, not a repo — is treated as dirty). Untracked paths are ignored, since the benchmark and setup-bench-repo.sh leave their own artifacts (shire.toml, .shire/bench.db) in the repo. Each phase restores exactly the bytes it captured before its first write, including when it aborts part way through a run. If the guard turns away every repo, the phase reports the skipped repos and exits non-zero rather than printing an empty result with a success status.

Setup

Run the benchmark repo setup script to clone and prepare the test repos:

scripts/setup-bench-repo.sh

This clones the three repos into ~/.cache/shire-bench/ and creates a shire.toml in each.

Running benchmarks

# Build the benchmark binary
cargo build --release --features bench --bin autoresearch

# Run build benchmarks (all repos)
cargo run --release --features bench --bin autoresearch -- --phase build

# Run query benchmarks (all repos)
cargo run --release --features bench --bin autoresearch -- --phase query

# Filter by repo size
cargo run --release --features bench --bin autoresearch -- --phase build --size small

# Point at a specific repo (must have a clean worktree for lifecycle/quality)
cargo run --release --features bench --bin autoresearch -- --phase build --repo /path/to/repo

Build benchmarks run 5 iterations (1 warmup + 4 measured) per repo. Query benchmarks run 100 iterations per query. Results are printed as JSON to stdout.

Environment notes

• Results vary by machine (CPU, disk speed, available memory)
• Close other applications for more stable measurements
• The warmup iteration primes filesystem caches and SQLite page cache
• Numbers in this document were captured on an Apple M-series Mac