Redb + Tantivy: The Atomic Dual-Engine at CameoDB's Core

What's new since this post (v0.3.8). The write path is now Redb first: the document and a WAL entry holding just its id commit together, then Tantivy indexes the document. Each Tantivy commit records the WAL position it covers, so recovery replays only what the index has not seen, and a shard reopens in milliseconds after any stop. Text fields can also name a language analyzer, with stemmers for seven languages.

The Hybrid Storage Philosophy

Most databases choose one storage model, relational tables for ACID guarantees, or search engines for rich query capabilities. CameoDB combines both behind one write path, delivering ACID-compliant key-value storage alongside full-text search without sacrificing consistency or performance.

The hybrid architecture combines two complementary storage engines, redb for durable KV storage with ACID guarantees, and Tantivy for full-text inverted indexes. Every write commits to redb first, and Tantivy follows the store as a derived view, so storage and search always agree on one source of truth.

Atomic Write Sequence

Each write operation follows a strict sequence. Redb commits first, and Tantivy follows:

// 1. Generate sequence ID (AtomicU64, lock-free)
seq_id = counter.fetch_add(1)

// 2. Begin redb transaction (ACID isolation)
txn = kv.begin_write()

// 3. Write to data table (complete JSON document)
data.insert(id, json_blob)

// 4. Append the document id to the WAL, in the same transaction
wal.insert(seq_id, id)

// 5. Commit redb transaction (the write is durable)
txn.commit()

// 6. Update Tantivy index (in-memory buffer)
writer.add_document(indexed_fields)

// 7. Periodic Tantivy commit, stamped with the WAL position it covers
writer.commit(wal_seq)

The sequence ID ensures monotonic ordering. The data table stores the complete JSON document, and the WAL records which document changed, in the same redb transaction. Once redb commits, the write is safe. Tantivy then indexes the document's fields, and its commits run on a steady clock, each one recording exactly how far through the WAL it has read.

ACID Guarantees Across Engines

CameoDB provides full ACID guarantees across the hybrid storage system:

Atomicity

All-or-nothing writes. The document and its WAL entry commit in one redb transaction, or not at all.

Consistency

Redb is the source of truth, and the search index follows it at every commit. A lookup by id sees a write the moment it returns, and a search sees it after the next commit.

Isolation

Concurrent operations don't interfere. redb's MVCC ensures safe parallel reads and writes.

Durability

With wal_sync on, the default, every write is fsynced when redb commits it. On bulk ingest that costs nothing measurable.

Multi-Tenant Architecture

CameoDB supports multiple isolated indices within a single storage instance. Each index operates as an independent tenant with its own data tables, search indices, and sequence counters. This enables multi-tenancy without resource contention between indices.

Data isolation is complete, per-index data_{index} and wal_{index} tables in redb, per-index indices/{index_name}/ directories for Tantivy. Sequence counters are independent per index, enabling parallel write scaling without lock contention.

Recovery and Consistency

On startup, CameoDB reads the WAL position stamped into each index's last Tantivy commit and replays only the ids above it, each from its committed row in redb. An index with nothing to replay is skipped without opening Tantivy at all, so recovery time tracks what was in flight when the process stopped, not how much data the node holds.

The recovery process is automatic and requires no manual intervention. It handles crash scenarios, power failures, and unclean shutdowns, and the search index comes back reflecting every committed write.

Performance Characteristics

The hybrid architecture delivers balanced performance across operations, measured with cameodb-bench on a single node:

Point Queries

~0.1ms inside the engine (one redb B-tree read)

Fast KV lookup by document ID, with no search index involved

Search Queries

5.75ms at p99

At concurrency 16 on an eight-core node, 6,300 searches a second

Bulk Writes

370,000 documents a second

Four shards, with fsync on every commit

Single Writes

65,000 a second

One document per request, with wal_sync off

The Takeaway

CameoDB's hybrid storage architecture combines the strengths of two specialized engines. redb provides ACID-compliant KV storage with fast point lookups and range queries. Tantivy provides rich full-text search with inverted indexes and relevance scoring. One write path, with redb as the source of truth, keeps both engines in step. The result is a database that delivers both transactional integrity and search capabilities in a single system, without the complexity of managing separate storage and search infrastructure.