Voice search with embeddings in the browser

Problem

The POS needed voice search for products in the sale flow, used with the company’s in-store hardware. During a sale, the search has to answer fast.

Decision

The flow has four steps: audio, transcription (gpt-4o-mini-transcribe), vector search for candidates, and product choice with gpt-4o-mini.

I decided to run the vector search in the browser, for three reasons:

• Speed. There is no network trip for each search.
• Cost. There is no embeddings service or vector database to pay for or maintain.
• The catalog was already there. The POS keeps it in the browser to sell offline.

The embeddings (all-MiniLM-L6) are computed with Transformers.js in a Web Worker. The index is an Orama index stored in the browser.

Result

It handles several products in one command, tracks usage with GA4 and loads only for companies that have it turned on. It is live in production, behind a per-company switch.

The search on this portfolio uses the same idea: open it from the header.

Limit

Transcription and product choice need a connection. Only the vector search is local.