fast-grep
Indexed regex search for codebases. 6–25× faster than ripgrep, sub-200ms per query, git-independent. Built so coding agents stop waiting for grep.
Indexed regex search for codebases. 6–25× faster than ripgrep, sub-200ms per query, git-independent. Built so coding agents stop waiting for grep.
Coding agents call grep on every reasoning step. Each call blocks the next thought. Across a session, slow search compounds into minutes of dead time per task — the single biggest latency leak in any agent harness.
Cursor's blog on indexed regex search lit the spark. They tied their index to git commits. We didn't: fast-grep watches any directory, kept fresh by an FS daemon, and works on non-git repos, generated artifacts, and unstaged changes. It's the fastest open-source indexed grep we've measured.
Five techniques combine to eliminate >99% of file I/O before the regex engine runs.
Every 3-byte window of every line is indexed as (document, line, byte offset). A query's required trigrams are intersected on (document, line) — the index answers "which lines", not just "which files".
Each trigram also has a bitmap of the documents containing it. Bitmaps are ANDed first (cheap, early exit); line postings are decoded only for the surviving documents.
Binary index files memory-mapped at query time. 17ms cold load regardless of corpus size — the OS pages in only the lists you touch.
Postings are delta-encoded with a flagged varint — ~2.9 bytes each instead of 16 — and encoded on the fly during the build, so the decoded index never sits in RAM.
Candidate lines are verified in parallel, seeking straight to each line's byte offset. Literals use memchr, alternations Aho-Corasick, regexes the Teddy SIMD engine.
Deep dive: Techniques · vs ripgrep · source code
Type a regex (or paste a real one) and see how fast-grep would decompose it into trigrams to look up in the index. Falls back to a full scan when there's no usable literal run.
The trigram decomposition shown here (literal vs alternation vs regex) mirrors what src/trigram.rs does; the real engine additionally intersects the resulting posting lists at line granularity.
The 5-stage path a query takes through fast-grep, with realistic numbers from the Linux kernel benchmark (EXPORT_SYMBOL over 81 690 files):
Steps 1–4 take ≈10 ms. Step 5 is the only stage that touches actual file bytes, and it reads candidate lines rather than whole files. End-to-end this query returns 197 matches in 197 ms — vs 1 553 ms for ripgrep.
Three zoom levels, from "where does this fit" to "what runs in the search hot path". Mermaid renders these in your browser.
Who uses fast-grep, and what does it talk to?
graph TB dev["👤 Developer
shell user"] agent["🤖 Coding agent
(Cursor, Claude Code, Aider)"] fgr(("fast-grep
CLI + optional daemon")) fs[(File system
any directory)] dev -- "fgr 'pattern' /path" --> fgr agent -- "tool-call: search" --> fgr fgr -- "mmap reads" --> fs fgr -- "watch + read" --> fs
Inside fast-grep there are two cooperating processes plus the on-disk index.
graph LR cli["fgr CLI
(Rust binary, single static)"] daemon["fgr daemon
(optional, FS-watcher)"] index[(Index files
mmap'd: postings, bitmaps,
lookup, meta.json)] fs[(File system)] cli -- "build / load" --> index cli -- "TCP localhost, token-authenticated
(flush before search)" --> daemon daemon -- "incremental
updates" --> index daemon -- "notify::Watcher
(debounced 3s)" --> fs cli -- "verify reads" --> fs
The internals of one indexed query, top-to-bottom in execution order.
graph TB cli["CLI / argument parser"] pat["Pattern decomposer
(extract literal runs &
alternation branches)"] load["Index loader
(lookup tables in RAM, postings + bitmaps mmap'd)"] ngram["Trigram extractor"] bloom["Roaring bitmap AND
(document tier)"] posting["Line posting intersection
(main + delta, tombstones filtered)"] verify["Line-level verify
(memchr / Aho-Corasick / regex)"] out["Output renderer
(grep / heading / compact / json)"] cli --> pat --> ngram --> bloom --> posting --> verify --> out load -.-> bloom load -.-> posting
The optional Metal GPU pre-filter (macOS, opt-in via FGR_METAL=1) sits in front of "verify" and runs literal scans on candidate lines as a Metal compute kernel.
Linux kernel 6.6 (81 690 files), Apple M1 Pro, warm cache. Numbers from the project's reproducible bench script.
| Pattern | fast-grep | ripgrep | Speedup |
|---|---|---|---|
TODO | 97 ms | 2 463 ms | 25× |
printk | 172 ms | 2 492 ms | 14× |
EXPORT_SYMBOL | 197 ms | 1 553 ms | 8× |
container_of | 344 ms | 2 440 ms | 7× |
static.*inline | 394 ms | 2 369 ms | 6× |
| Pattern | fast-grep | ugrep | Speedup |
|---|---|---|---|
EXPORT_SYMBOL | 197 ms | 1 898 ms | 9.6× |
TODO | 97 ms | 599 ms | 6.2× |
static.*inline | 394 ms | 1 595 ms | 4.0× |
printk | 172 ms | 645 ms | 3.8× |
container_of | 344 ms | 656 ms | 1.9× |
| Full build | ~60 s (one-time) |
| Incremental update | <1 s for 10–100 files (75× faster than rebuild) |
| Index load (mmap) | 17 ms |
| Index size | ~2.9 GB postings + ~170 MB bitmaps (79 406 indexed files, line-level postings) |
Where the project sits today, and where we want it to go. Numbers reflect cargo test --release output as of the latest release.
What a healthy pyramid for this project looks like — gaps to fill, ranked by ROI:
| Tier | Now | Target | Gap (why it matters) |
|---|---|---|---|
| Unit | 33 | ~80 | Coverage holes in persist.rs, daemon.rs, the output_matches/highlight_into renderer (the recent v0.3.1 work has zero unit coverage). |
| Property | 0 | ~10 | Invariants of the regex decomposer: literal runs of length ≥3 should always intersect-match the input; alternation splits should round-trip. proptest is the right tool. |
| Integration | 9 | ~25 | Daemon lifecycle (start → fs change → flush before search), incremental update against committed-then-modified files, --type filter combinations. |
| CLI snapshot | 0 | ~15 | Lock the user-facing TTY output (grouped/colored) and piped output (path:line:content) with insta snapshots so we don't silently regress the rendering. |
| Fuzz | 0 | 1 target | cargo-fuzz on the verifier: random pattern + random byte buffer should never panic and should match the regex crate's own behavior. |
| Bench (regression) | baseline JSON | CI gate | Wire scripts/bench.sh into a GitHub Action that compares against benches/baseline-v0.3.1.json and fails the build if any pattern regresses by >15%. |
All channels we control. Community packaging (apt, Fedora, Arch, MacPorts, Chocolatey…) welcome — see README.
# Cargo (any platform with a Rust toolchain)
cargo install fast-grep
# Prebuilt binary via cargo-binstall
cargo binstall fast-grep
# Homebrew (macOS / Linux)
brew install gmilano/fast-grep/fast-grep
# Scoop (Windows)
scoop bucket add fast-grep https://github.com/gmilano/scoop-fast-grep
scoop install fast-grep
# Debian / Ubuntu — .deb attached to every release
curl -LO https://github.com/gmilano/fast-grep-rust/releases/latest/download/fast-grep_0.3.1-1_amd64.deb
sudo dpkg -i fast-grep_*_amd64.debThen:
# Build the index once (auto-built on first search if missing)
fgr index /path/to/repo --output .fgr
# Search — sub-200ms on cached queries
fgr "EXPORT_SYMBOL" /path/to/repo --index .fgr
# Watch + auto-update on file changes
fgr daemon start /path/to/repo --output .fgr