A few weeks ago I bought a Plaud, one of those small recorders that transcribe and summarise everything you say. And I have to say it: what I'm getting out of it is incredible. Meetings, classes, ideas that come to me while walking… everything gets saved and transcribed without lifting a finger.

But I soon hit a limit. Plaud does have AI generation, yes, but it doesn't use the most powerful models. The summaries are fine to get by, but they don't connect one conversation with another or understand the context of your life. You end up with hundreds of loose notes instead of a memory.

So I set out to build my own.

Second Brain, running locally

Second Brain is a project that runs on my computer. I built it with a Claude Code agent working in the background, and it has one goal: to optimise how I capture and make use of my notes.

Main dashboard of Second Brain, the external memory I feed with my Plaud recordings — The Second Brain dashboard («External memory»), shown here on an empty instance so no personal data is exposed.

This is how it works:

It collects the notes. It pulls the Plaud transcripts automatically.
It understands them. A powerful model reads each recording and extracts people, projects, topics, tools, goals and the relationships between them.
It builds a graph of my life. All those entities are connected in a graph: who shows up with whom, which project touches which topic, which ideas keep coming back.
It generates memories. I can ask it in natural language ("what did I talk about with X regarding project Y?") and it answers with the sources and the exact moment in the recording.
It dreams. It has automatic dreaming: without me doing anything, it goes over what it has stored and surfaces new memories and connections.
It suggests actions. It tries to turn what I say into concrete steps: follow-ups, pending tasks, questions to review what I learn in class.

What matters most to me: no making things up

A second brain that invents things is worse than having none at all. That's why every entity, relationship or action it extracts carries a verbatim quote from the transcript, and the system checks that the sentence really exists. From any piece of data I can jump back to the original recording.

And the actions are only suggestions. Nothing runs on its own: I decide what to accept, what to complete and what to discard.

A quote existing doesn't prove the interpretation is right. That's why the dashboard always lets you go back to the source.

Private by design

Everything lives on my machine: transcripts, database and graph. Nothing is uploaded to any repository. On top of that, recordings I mark as sensitive are kept out of general searches and out of the graph.

It also has its own MCP server, so I can query my memory from other agents, and it exports everything to Obsidian-compatible Markdown.

The lesson

Capture hardware is a solved problem: a small device records everything and transcribes it well. What makes the difference is what you do with that data afterwards. With a good agent and an afternoon of work you can turn a pile of loose notes into something that looks a lot like an external memory.

If you have a Plaud (or any recorder with transcription) and want to build something similar for yourself or your team, get in touch.