A voice companion that remembers

A voice loop is brutal about latency. The user finishes speaking, and every millisecond until the first synthesized word is a pause they can hear. Drop a memory read into that gap, a vector-DB round-trip, then an LLM re-rank to decide what's relevant, and you've added a beat of silence to every turn. The assistant sounds like it's thinking when it should already know.

Korely's read path has no generative model on it: get_context embeds the query once to rank, then assembles the user's active facts most relevant to it plus their most relevant memories within a token budget. Retrieval, not generation. That keeps the lookup small enough to sit before the first spoken token, and you should time it from your own region to see what it adds to your loop. Korely publishes no latency guarantee. This cookbook walks the pattern at the call level. Terms such as end user, fact and context block are in the Glossary.

The snippets use the Python SDK (pip install korely-memory). The same calls work over the REST API and the Node SDK.

The shape of a voice turn

Three stages, each from a different vendor. Speech-to-text (Whisper, Deepgram) turns the mic into a transcript. Your agent reasons and replies. Text-to-speech (ElevenLabs, and the rest) turns the reply into audio. Korely is none of those, it's only the memory, and it has exactly one job on the critical path: hand the agent what it knows about this user before the model runs.

from korely_memory import Korely

import time
korely = Korely(api_key="kor_live_...", region="eu")

# STT + TTS are your choice: placeholders here, Korely is only the memory.
transcript = stt.listen()        # Whisper / Deepgram -> text
USER = "user-amelia"             # the end user this companion serves

1. On the critical path: read context before the model runs

The moment the transcript lands, assemble what's known about the user into a prompt-ready block. One call. get_context returns the active facts most relevant to the question (at most ten) plus the most relevant memories, already trimmed to a token budget, and there is no second LLM hop deciding relevance. Measure it, as in the snippet, to see whether it fits before your time-to-first-token.

t0 = time.perf_counter()
ctx = korely.get_context(
    query=transcript,            # "remind me what my doctor said about coffee"
    user_id=USER,
    token_budget=600,            # keep it tight: this rides the latency budget
)
print(f"memory read: {(time.perf_counter() - t0) * 1000:.0f} ms")  # measure it in your setup

# ctx.context -> a Markdown block to prepend to the system prompt
#                (below, the note for the model that opens it and the line on
#                dates are shortened, and the stable, volatile and stable_hash
#                fields of the response are left out)
# ctx.tokens  -> how many tokens it actually used
# ctx.sources -> the fact and memory ids it drew from

reply = agent.respond(system=ctx.context, user_turn=transcript)
tts.speak(reply)                 # ElevenLabs / your TTS -> audio out

{
  "context": "_The facts below are a compact profile of the user; the memories are the verbatim source of truth. Ground your answer in this context. [...] do not abstain when it is present._\n\n## Known facts\n- The user caffeine_cutoff noon (since 2026-11-01)\n- The user likes decaf oat flat white at the new cafe on Rua do Carmo (since 2026-11-01)\n- The user lives_in Lisbon (since 2026-11-01)\n\n## Relevant memories\n_A time in square brackets after a relative expression [...] was already resolved from the day that memory was written [...]. The date at the start of a memory [...] is the day it was said._\n- [2026-11-01] Amelia's doctor moved her caffeine cutoff to noon, it used to be 2pm. She lives in Lisbon. She likes the decaf oat flat white at the new cafe on Rua do Carmo.",
  "tokens": 353,
  "sources": ["fct_...", "fct_...", "fct_...", "mem_..."],
  "degraded": false,
  "degraded_parts": []
}

This is real output for one memory written a few minutes earlier, with the ids shortened and the two notes for the model shortened as before. The facts are dated with the day the memory was written, because that write carried no timestamp. The fact names (caffeine_cutoff, lives_in) are chosen by the extractor and will differ for your text.

The agent now speaks with the user's history already in hand, the doctor's advice and the decaf preference, and has no need to say "let me look that up."

Why there is no model on the read path. Only one embedding of the query, made on Korely's own server. get_context assembles facts plus memories and does not generate text. A vector-DB round-trip plus an LLM re-rank adds a second model call to every turn; this does not. Facts that were closed (a "drinks espresso all day" fact, once a newer fact replaced it) are left out, so the model gets what is true now, not a pile of candidates to sort through out loud.

STT (your choice)

Mic -> transcript

Whisper / Deepgram
The user stops speaking

Korely

get_context(query=transcript)

Active facts + relevant memories
One query embedding, no generative model

Agent + TTS

Grounded first token

Model speaks already grounded
Time the lookup in your loop

2. Mid-turn: pull one specific memory

When the agent needs a particular past detail rather than the whole context block, search the user's memories directly. Scoped to the user, ranked by relevance, no generative model in the path.

hits = korely.search(
    "cafe Rua do Carmo decaf",
    user_id=USER,
    limit=3,
)
# search() returns the list of hits directly, not a page object.
# each hit: {id, score, snippet, user_id, agent_id, metadata}

[
  {
    "id": "mem_a8bf...",
    "score": 0.4751,
    "snippet": "Amelia's doctor moved her caffeine cutoff to noon, it used to be 2pm. She lives in Lisbon. She likes the decaf oat flat white at the new cafe on Rua do Carmo.",
    "user_id": "user-amelia",
    "agent_id": null,
    "metadata": {"channel": "voice"}
  }
]

That is real output for the memory above, with the id shortened. The score is how close the memory is to the words of the query: a keyword-style query like this one scored 0.48, and a question phrased like the memory scores higher. Compare scores within one query, not against a fixed threshold. In one run the search call took about 270 ms from a laptop, which is a network round trip plus the query embedding, not a promise.

3. Off the critical path: capture the turn

After the assistant has spoken, once the latency-critical work is done, write the turn down. Run it in the background so it doesn't block the next spoken word, and the user is already hearing the reply while it runs.

korely.add(
    "User said her doctor moved the caffeine cutoff earlier: "
    "no coffee after noon now, not 2pm.",
    user_id=USER,
    metadata={"channel": "voice"},
)
# add() accepts a plain string or a chat-message list [{role, content}, ...].
# Returns once the memory is stored; fact extraction runs after, server-side
# (usually within about ten seconds).
# To keep the write off the voice loop, run it in a background thread, or use
# AsyncKorely and schedule korely.add(...) as a task.

{
  "id": "mem_77e1",
  "content": "User said her doctor moved the caffeine cutoff earlier: no coffee after noon now, not 2pm.",
  "user_id": "user-amelia",
  "agent_id": null,
  "run_id": null,
  "metadata": {"channel": "voice"},
  "client": "korely-python/0.1.28",
  "kind": "note",
  "status": "processing",
  "created_at": "2026-06-16T18:40:11+00:00",
  "updated_at": null,
  "facts": []
}

Note facts comes back empty here, extraction hasn't run yet. The new (subject, predicate, object) triple (for example user-amelia avoids "caffeine after noon") lands a few seconds later, server-side. If an older "after 2pm" fact is open, a model judge can close it. The judge can miss, which is why the write states the new cutoff in the present tense ("no coffee after noon now"). When you do need to confirm what was extracted, read it back explicitly rather than expecting it inline:

# moments later, off the voice loop: read the extracted facts back
mem = korely.get("mem_77e1")
# mem.facts is now populated (an empty list means no fact was extracted)
# you can also read the end user's facts directly:
facts = korely.get_facts(user_id=USER)                              # open facts
history = korely.get_facts(user_id=USER, include_invalidated=True)  # plus closed ones

Once extraction has finished, get_context serves the corrected cutoff. If the judge closed the "2pm" fact, the block no longer lists it; a memory that mentions "2pm" can still appear under the relevant memories, so read the block as a whole.

One honest caveat: capture is the asynchronous half. korely.add returns as soon as the memory is stored and runs fact extraction off the latency-critical path, so a turn's typed facts land usually within about ten seconds of the write, ready for a later turn's get_context, not inside the same call. Write turns in the background once the assistant has spoken. And to be clear about the division of labour: STT and TTS are yours (Whisper, Deepgram, ElevenLabs and the rest), Korely is only the memory that loads in time.

Where to go next

get context is the call on the critical path in step 1, its full options and budgeting behaviour; Temporal facts explains how a newer fact can close an older one, such as "after 2pm" becoming "after noon", so the read path serves what's true now; and the multi-session research cookbook shows the same read/write split when latency isn't the constraint and the emphasis is point-in-time replay.