Fast by Design: Benchmarking CameoDB

What’s new since this post (v0.3.8). Commits moved to a steady clock, which made bulk ingest up to 6.6× faster. A search now fetches only the documents on the page it returns, a shard reopens in milliseconds after any stop, and every search answer carries stats.timings_ms, showing where its time went. The numbers below are the current ones.

Lean by Design

When you're processing millions of documents, every copy matters. Traditional databases move data through layer after layer of buffers and intermediate structures, each adding latency and memory pressure. CameoDB keeps the path short.

Built with Rust 2024 Edition, CameoDB uses the language's ownership model to pass data along instead of copying it. A bulk write hands its batch from the request to the shards without duplicating it, and the write-ahead log records only a document's id, because the document itself is already in the same transaction. Fewer bytes to copy, and fewer bytes to fsync.

Hybrid Storage: Two Engines, One Pipeline

CameoDB's performance comes from its hybrid architecture, combining redb (ACID-compliant KV store) with Tantivy (full-text search engine). Each write follows one sequence, with redb first:

// 1. Generate sequence ID (AtomicU64, lock-free)
seq_id = counter.fetch_add(1)

// 2. One redb transaction: the document and its WAL entry
txn = kv.begin_write()
data.insert(id, json_blob)
wal.insert(seq_id, id)
txn.commit()                  // durable

// 3. Tantivy follows the store
writer.add_document(indexed_fields)
writer.commit(wal_seq)        // on a steady clock

The key insight: Tantivy stores only indexed fields. The complete JSON document lives exclusively in redb. When you search, each shard answers with keys first, the results are ordered and windowed, and only the documents that make the page are fetched from redb. Smaller indices, faster searches, and zero data duplication.

Commits on a Clock

Committing an index is expensive, and doing it on every write would cost throughput. CameoDB commits the search index on a steady clock: once the oldest uncommitted write has waited commit_interval_ms while writes keep arriving, and once more when they settle. A write is durable the moment redb commits it and searchable after the next index commit, and no write ever waits for one.

Moving commits from a count to a clock made bulk ingest between 1.8 and 6.6 times faster, because a busy shard writer no longer stops to wait on the indexer.

Cache Sized per Shard

Each shard's redb cache is sized by tier from the data it holds and kept within its share of the node's memory, so a node carries many shards comfortably with no manual tuning. After any stop, planned or not, a shard reopens in milliseconds: every commit records redb's allocator state, so there is nothing to rebuild on the way back up. A 4 GB shard file opens in about 7 ms.

Benchmark Results: The Numbers

These are end-to-end figures from cameodb-bench, the harness in the repository: client-observed over HTTP on a single node, with settings tuned for throughput.

Bulk Ingest

370,000 documents a second

Four shards, fsync on every commit

Single Writes

65,000 a second

One document per request, wal_sync off

Search Throughput

11,600 searches a second

On 15 cores, scaling with every thread you give it

Search Latency

5.75ms at p99

Concurrency 16 on an eight-core node, 6,300 searches a second

Point Queries

~0.1ms inside the engine

One redb B-tree read by document ID

Two results stand out. Durability is free on bulk ingest: at saturation the node measured 370,932 documents a second with fsync on and 365,382 with it off. And the node reaches its plateau using about a quarter of its cores, so the way to go faster is larger batches, not a bigger machine.

Async-Sync Isolation: Blocking Without Blocking

All redb and Tantivy I/O happens inside tokio::task::spawn_blocking. This is critical: storage operations are inherently blocking (disk I/O, B-tree traversals, index segment merges). By offloading to a dedicated blocking thread pool, CameoDB's async runtime stays responsive. No thread starvation. No latency spikes from blocking the event loop.

The Takeaway

A short data path, hybrid storage, commits on a clock, per-shard caching and async-sync isolation aren't isolated optimizations. They're one architecture where each part amplifies the others. Rust's ownership model keeps copies out of the path, the split-storage strategy keeps indices small, and the commit clock keeps writers moving. The result: hundreds of thousands of durable writes a second and millisecond searches, measured end to end.