JARVIS · Dorian Bramarov

Real app · made-up data

The real HUD, running on made-up data. My notes as a living graph in the middle, system dials on the left, and the ring, to-dos and voice budget on the right.

Why I built it

Mostly for fun, and to learn. I wanted to get hands-on with speech to text, text to speech, tuning a voice and a personality, and permissions: what an assistant should be allowed to do on its own. And I wanted it to look impressive.

It runs on my PC and starts when I sign in. I say "Hey Jarvis", and it handles notes, to-dos, reminders and quick commands.

Key decisions

1 · Speed is part of the design

Speech to text and text to speech both stream, so JARVIS starts answering while it's still working. A pause that feels fine in a chat window feels broken out loud.

"Hey Jarvis"wake word
Speech → textDeepgram, streaming
Claude + toolsnotes · to-dos · reminders · commands
Text → speechElevenLabs, streaming
Reply

Face: listening → thinking → confirming (if it changes something) → speaking

Simplified flow. Streaming on both ends keeps it conversational, and the face shows what JARVIS is doing at every step.

2 · Show what it's doing

Voice has no screen by default, which is why it's hard to trust. JARVIS has a face that shows six states: idle, listening, thinking, speaking, confirming and refused. It moves with the real audio, so you can tell what it's doing from across the room.

Real app · made-up data

The other view: JARVIS as a face, made of a few thousand points of light, going from listening to thinking to speaking. In the app, its mouth moves on the real audio of its voice.

3 · Permissions before features

Reading is cheap; acting isn't. Actions that change things can wait for my OK, and an optional voice check means only my voice can trigger them, so someone talking in the room can't run commands. When it asks, it reads the action back from the action itself, not from the model's description, so what I approve is exactly what runs. Once outside text like an email is in the conversation, even saving a memory waits for a yes. And its Google access is limited to the narrowest permissions that cover what it does.

4 · A voice of its own

Its voice was designed from a character description, not cloned from a real person. I tuned how it sounds and responds (the voice, the wake word, the timing) and its personality and instructions, until it felt like someone without being anyone.

Designing the HUD

The HUD is designed for a second monitor. Before writing any code I explored it as mockups: three dense "command center" layouts and a fourth, Nexus, with my notes at the center as a living graph. I even prototyped how the pulses along the links should move, so the graph looks like real nerve signals instead of a screensaver.

A – CCommand centerThree data-dense panel layouts: calendar, mail, system stats, to-dos and reminders around the assistant.

D · builtNexusMy notes as a living graph at the center, with pulses moving along the links.

Four directions as mockups before any code. Nexus won, and it's what runs today.

What I'm proudest of

How it looks: sleek and visually impressive. The centerpiece is a custom node graph of my notes, with pulses that travel along the links. And a face that shows what JARVIS is doing, listening, thinking or speaking, with its mouth moving on the real audio of its voice. Under all of that, it feels like a real conversation.

What I learned

Speed is part of the design. And plan before you build: mockups first, and security first. Deciding what it's allowed to do is a lot easier before the code exists than after.

In daily use on my PC since September, packaged as a desktop app with more than 200 automated tests.