Why I've been building Nastalgic
Augmented memory has been on my mind since AI got useful. Here's why I've been building Nastalgic — and why security and extensibility were the lines I wouldn't cross.
The blog has been empty for a reason. I’ve been heads-down on the thing the site keeps pointing at.
Augmented memory has been stuck in my head ever since AI started actually feeling useful, somewhere around the end of 2022. Once I watched a model reason competently inside a single conversation, the obvious-but-unsolved part was the next conversation. Why did it forget? Why did the prompt-stuffing tricks people were shipping feel like a workaround instead of a design? I posted about this on X a couple of years back — same hunch, fewer answers. The hunch eventually turned into Nastalgic.
The short version of what it does is on the homepage: a 4-stage extraction pipeline (Resolver → Facts → Inference → Graph) that turns conversation into a queryable knowledge graph. I’ll go deep on the pipeline in another post. This one is about the two decisions I made early and never let myself walk back — security and extensibility — because if I had compromised on either, the project wouldn’t be worth shipping.
Security came first, not last
There’s a category of memory tool that solves the storage problem and waves at the isolation problem. I didn’t want to build that.
Vault isolation in Nastalgic isn’t a flag you flip in a config. It’s the architecture. Every account gets a vault — a separate Postgres database, a separate Qdrant collection, a separate Redis instance, a separate worker. The free tier is logically isolated on shared infrastructure (vault-scoped queries, per-vault credentials decrypted on demand). Paid plans get dedicated database and vector store. Enterprise gets dedicated infrastructure end to end. Same code path, different physical placement.
A few things that fall out of that one decision:
- Cross-vault queries don’t compile. The connection manager routes by vault ID on every request. There is no static database URL for vault data. If a code path doesn’t know which vault it’s in, it doesn’t run.
- BYOK is on every plan. Bring your own provider key for the extraction worker and your traffic to the LLM provider stays under your account, on your terms. The plaintext key isn’t kept around after encryption.
- Auth is JWT RS256 with refresh tokens and personal access tokens. Tokens are vault-scoped — a token for vault A literally cannot read vault B. Token blacklisting is Redis-backed, and a Redis outage fails closed.
- Provenance is structured. Every fact in the graph carries a typed evidence edge back to the source message that produced it. Nothing in memory is unsourced. If a customer ever asks “where did this come from,” the answer is one query away.
None of this is exotic. It’s what you build when you decide on day one that “trust us” is not a security architecture.
Built to extend
There’s a category of memory tool that bolts on every new provider, every new model, every new use case until the whole thing creaks. I didn’t want to build that either.
Extensibility in Nastalgic isn’t a roadmap promise. It’s wired into the surfaces that already exist.
The system is five small services — management, interaction, worker, spaCy, frontend — sharing one library, with boundaries that are deliberate. The LLM layer is a single abstract base class with one contract method; providers (Anthropic, OpenAI, Gemini, Ollama) live in independent files behind a pure-function factory. Adding a fifth provider is a new file and one branch. The extraction pipeline is built on the same shape: every module implements run(context) -> StageResult, reads prior stages through context, and never mutates shared state. Slotting in a sharper resolver or a multimodal fact extractor doesn’t touch anything else. The public API is REST with auto-generated Python and TypeScript SDKs off a single OpenAPI spec, so the client surface follows the server contract on its own.
I don’t want to be the framework that takes over your agent loop. And I don’t want my own future self stuck inside a monolith.
It has been a lot of fun
That part I didn’t expect. I’ve been writing software for a long time and most projects feel like a march. This one has felt like a puzzle the whole way through — coreference resolution one week, cross-encoder reranking the next, then back to figuring out how to make per-vault Redis routing not melt under load. I’ve learned more in the last twelve months than in any year of my career, and a lot of it is NLP and information-retrieval research I used to read for fun and never get to use.
I dogfood it every day. It remembers what I told it about the codebase, and it pings me when I’m contradicting something I said three weeks ago. That feedback loop alone has been worth the build.
July
Production launches in July. The site is up, the waitlist is open, the pipeline is running on real conversations daily. There’s plenty left — Stripe, hardening, the legal track — but the shape of the thing is real, and I’m itching for the day I get to flip the switch.
If you want the build-out posts as they land, the waitlist is the way. More soon.
— Rob