← Back to Blog
Research

What Retrieval Costs on the Board: Ventuno Q Benchmarks

•20 min read•Tara Khani, Neel Patel

These numbers are from Ventuno Q. The board has 15,284 MB of RAM. Every run uses the same setup: random unit vectors, 1536 dimensions, seed 42, top_k = 5, five warmup queries, and 50 timed queries.

Moorcheh stores one bit per dimension, so a 1536-dimension vector is 192 bytes. Search is an exact scan of those bits, and the built-in rerank of the shortlist is already inside that search time. The comparison index is HNSW on the original float32 vectors, scored with cosine. The HNSW numbers in the charts are the graph only — they do not include a reranker.

Moorcheh Edge (in-process)

One bit per dimension (192 B per 1536-d vector). Exact bit scan plus built-in shortlist rerank in the same search interval. One JSON store file — no separate graph build.

HNSW on float32 (comparison)

Insert vectors, then a dedicated graph-build step. Cosine on original floats. Timed search is graph only. A second-stage ms-marco-MiniLM-L-6-v2 file is about 80 MB on disk; we did not time it.

Side-by-side at scale

At 100k items, Moorcheh’s on-disk file is about 40× smaller than the settled HNSW store (29 MiB vs 1.3 GiB). Moorcheh upload was 67 s; HNSW needed about 3.3 minutes just to build the graph after insert. Median search is in the same ballpark (6.45 ms vs 4.22 ms graph-only); the search chart stacks a fixed 12 ms planning rerank (orange) on the measured HNSW bar (red) at every scale so end-to-end HNSW can be compared to Moorcheh’s measured rerank-inclusive latency. That orange slice is ms-marco-MiniLM-L-6-v2, about 80 MB on disk. Moorcheh does not ship a separate reranker file.

At 1M, the gap shows up on disk, index build, load path, and process RAM — see the charts below.

On-disk store size by catalog scale (Ventuno Q)

HNSW figures are settled store size (vectors + graph + WAL footprint varies during build). Moorcheh is a single JSON file per run.

Median search latency (50 timed queries, ms)

Green is measured end-to-end (rerank included). Each HNSW column is one bar, same width top to bottom: red is the measured graph, orange on top is a +12 ms planning pass for ms-marco-MiniLM-L-6-v2 (about 80 MB on disk; not timed). The number above the column is the stacked total.

Moorcheh on Ventuno Q

In-process. Process RAM is the retrieval process after search. Store size is one JSON file (a bit larger than raw bits because each row carries an id). Raw bits alone would be about 1.8 MiB at 10k, 18.3 MiB at 100k, and 183 MiB at 1M; the measured files are 2.9 MiB, 29.3 MiB, and 294 MiB with ids included.

At 10k, upload took 2.2 s and median search was 0.89 ms (min 0.32 ms, p95 1.90 ms) with 58 MB process RAM. At 100k, upload was 67.0 s and median search 6.45 ms (min 3.67 ms, p95 7.80 ms) with 173 MB process RAM.

The 1M upload is recorded ingest at 9,467 s (~2 h 38 min, ~106 vectors/s). There is no graph build step. Search and RAM at 1M are from loading that store again: 50 queries, process RSS flat at 737 MB — search does not grow RAM.

Opening the 100k file takes 0.40 s; the first search after that is 4.9 ms and the next is 4.0 ms (~77 MB process). Opening the 1M file takes 1–2 s; search then matches the medians in the chart.

Moorcheh median search on Ventuno Q (includes shortlist rerank)

At 1M: min 11.63 ms, mean 14.95 ms, p95 17.95 ms over 50 timed queries after warmup. Upload at 1M was 9,467 s (~106 vectors/s); index build 0 s.

Moorcheh process RAM vs Ventuno Q (15,284 MB total)

At 1M vectors, ~14.5 GB remains for embedder, LLM, Whisper, Piper, and vision — vs ~7.6 GB headroom if HNSW holds 7.4 GB for retrieval alone.

If the index is HNSW

Same board, same 1536-dimension vectors: insert, then a separate step that builds the graph. Float vectors stay on disk; cosine runs on those floats. At a million chunks, production stacks often add a second pass to rescore the graph’s shortlist — we did not run that; it would sit on top of the search times below.

At 10k, insert was 1.56 s and graph build 14.7 s; median search 3.41 ms. At 100k, insert was 14.1 s and graph build about 198 s (~3.3 min); median search 4.22 ms. Host memory during insert and build moved from about 850 MB toward 990 MB at 10k and 1,346 MB at 100k — whole-machine figures, not the graph alone.

At 1M on a clean shard, insert was 288.9 s (~3,461 vectors/s) and graph build 10,658 s (~2 h 58 min). After reload without new insert or build, the shard on disk was 13.2 GiB (83 MB HNSW graph, 5.8 GB float vectors in segment storage), process RAM 7.4 GB, open 2.5 s, first search 7.6 s, next search 6.6 ms, warmed median 5.28 ms.

During graph build, disk can temporarily hold a second copy of the floats plus a large write-ahead log. We observed about 19 GB on disk mid-build on this board before the store settled to 13.2 GiB.

HNSW index build time after insert (Moorcheh: 0 s — no graph step)

At 1M vectors, insert took 4.8 min; graph build alone took 2 h 58 min on the same Ventuno Q shard.

At 1M vectors: load path and process RAM

Time to ready-to-search

Process RSS with store loaded

After HNSW warm-up, median search is 5.28 ms (graph only). Moorcheh median 14.84 ms includes rerank — no second model pass.

At 1M vectors: where the clocks diverge

  • Index buildMoorcheh0 sHNSW2 h 58 min (10,658 s)
  • Load / first usable searchMoorchehOpen file 1–2 s, then search as belowHNSWOpen shard 2.5 s; first search 7.6 s (cold floats)
  • Search median (warmup done)Moorcheh14.84 ms (scan + rerank)HNSW5.28 ms graph + ~12 ms est. rerank ≈ 17.3 ms
  • RerankMoorchehBuilt into the search intervalHNSW~12 ms planning (ms-marco-MiniLM-L-6-v2, top-5, ~80 MB on disk); graph-only timing was measured

Moorcheh has nothing to build after upload: the bits in the 294 MiB file are what search scans. HNSW inserts in 4.8 minutes at 1M, then spends 2 h 58 minutes building the graph. After reload, opening the shard is 2.5 s, but the first search is 7.6 s while ~5.8 GB of floats are read; the next search is 6.6 ms and the warmed median is 5.28 ms.

These scores are not the same ranking problem. HNSW cosine ranks original floats and stops at the graph. Moorcheh ranks bits and the published interval is finished retrieval. A smaller store is not a claim that both orderings match.

RAM left for the rest of the stack

Ventuno Q has 15,284 MB of RAM. Moorcheh’s retrieval process after the store is loaded is what the headroom chart shows. Everything else — on-device embedding, a language model, vision, voice — comes out of what that process does not take.

At 1M vectors, retrieval is 737 MB of process RAM and a 294 MiB file. The other ~14.5 GB is available for the models that turn the store into a product. Model sizes in the planning chart below are published weight and runtime figures (plus one embedder footprint we measured on a smaller board); they are not a second Ventuno process capture except for the 737 MB retrieval number.

A typical multimodal stack beside that catalog might budget roughly: BGE-small embedder (~133 MB weights, ~676 MB process on UNO Q), Qwen 2.5 3B Q4 (~2.1 GB weights) or 7B Q4 (~4.7 GB), Whisper small (~466 MB on disk, ~750 MB resident), Piper (~25 MB voice file, a few hundred MB resident), and a YOLO-class 1080p stream (tens of MB engine, on the order of 100–300 MB runtime). Summed with 737 MB retrieval, a 3B stack lands near 5.0 GB on the 15,284 MB board; swapping in 7B weights still leaves headroom. Context cache on the LLM grows with prompt length. On disk beside the 294 MiB retrieval file, plan for those model artifacts separately — the HNSW shard alone is 13.2 GiB before any of them. An HNSW stack that rescores the shortlist also keeps ms-marco-MiniLM-L-6-v2 on disk, about 80 MB. That file is the cross-encoder behind the +12 ms planning bar. Moorcheh’s built-in shortlist rerank does not add it.

Example multimodal stack on the 15,284 MB board (1M-vector catalog)

737 MB retrieval + ~676 MB embedder + Qwen 2.5 3B Q4 + Whisper small + Piper + YOLO-class vision ≈ 5.0 GB. Same pieces beside HNSW (7.4 GB retrieval) ≈ 14.3 GB. Figures are planning arithmetic, not a single captured process list on Ventuno.

Reference architecture — LaneOrder on Ventuno Q

The planning sum above is abstract; LaneOrder is one concrete layout on the same board. Multimodal menu and catalog data (text, image, audio, video) is chunked and embedded into Moorcheh. The production catalog index and Moorcheh search sit in the ~737 MB retrieval budget at million-vector scale. Live flows — cart, kiosk, ticket, and spoken reply — are latency-bound through GenieX and a local Qwen-class model on top of the top-k hits. Upload, reindex, eval, and scenario pricing stay on a separate throughput-bound path so ingest does not compete with sub-15 ms search during service hours.

Reference architecture — LaneOrder on Ventuno Q: multimodal text, image, audio, and video through Moorcheh to GenieX Qwen

Figure: LaneOrder on Ventuno Q — multimodal text, image, audio, and video → Moorcheh (embeddings, catalog chunks, menu index, indexing, search, catalog store) → GenieX Qwen on the latency-bound semantic drive-thru path (cart, kiosk, ticket, reply); catalog ops (upload/reindex, eval, scenario prices) on the throughput-bound path.

The 1M catalog we measured is 1536-dimensional. BGE-small (384-d) and BGE-base (768-d) produce smaller stores when the embedder matches; the 737 MB figure is for 1M × 1536 on this run.

Use cases on the board

Spoken answers over a million chunks: Whisper transcribes, the embedder builds a query, Moorcheh returns top-k in about 15 ms median at 1M, a local LLM answers from those passages only, Piper speaks the result. The wait is usually the LLM and speech, not the scan.

Live camera plus catalog: a small detector labels the frame; that label is embedded and searched against product or part vectors; the LLM narrates the match. The archive stays a few-hundred-megabyte file, not a video warehouse.

Hands-free field assist: microphone and camera stay up together; speech and vision both query the same million-vector manual offline without shutting down retrieval.

Voice can run one step at a time on tighter boards (record → transcribe → search → speak). On Ventuno, the planning sum fits with those pieces resident at once — the headroom the 737 MB retrieval process leaves.

Air-gapped retail voice and vision kiosks

Retail kiosks on Ventuno Q can run fully offline — no store Wi‑Fi or cloud APIs. A shopper asks where to find a replacement filter and how to install it; Whisper transcribes, Moorcheh searches across a large catalog of SKUs, schematics, and indexed content in about 15 ms median, a local LLM produces grounded steps, and Piper reads the answer while the UI shows the retrieved diagram.

Autonomous mobile robots and spatial video memory

AMRs can use Ventuno Q as an integrated compute brain: embed keyframes, depth cues, and obstacle logs into Moorcheh instead of shipping video to the cloud. When a pallet is missing or a corridor is blocked, the robot queries when and where an asset last appeared in its indexed stream — on the edge.

Hands-free field assist at scale

Technicians carrying rugged kits can host a million-plus schematics, wiring diagrams, and maintenance clips in Moorcheh. Point the camera at a component and ask a spoken question; Moorcheh cross-searches the library in under 15 ms and the local LLM narrates the matching diagram or clip through a headset.

See also Moorcheh Edge on Arduino.

References and further reading

Moorcheh

Qualcomm

Arduino

bash
pip install moorcheh-edge
moorcheh-edge up -y
moorcheh-edge status

Build this architecture today.

Get your API key and start building agentic memory in under 5 minutes.

Get API Key