Introducing Breadcrumb, an AI plugin that records everything and learns how you work
TL;DR
Breadcrumb is a Mac app that has two sides to it. It's an AI context management system, and it also records everything you do. It plugs in to all your AIs, it's local, encrypted and actually free.
Compounding
When it's architected well, the combination of always-on screen recording and deep context management is powerful. And the benefits compound.
When I say context management I’m not really talking about a memory system. I’m talking about a system that enables you to tightly control your AI, and give it access to everything it needs to know you well, and do good work for you.
Controlling Your AI
Anyone who’s spent any time working with AI will tell you it does annoying things ALL THE TIME and you can never guess what the next annoying thing will be.
The only sure way to stop an AI from drifting is to tell it your rule exactly when it needs to hear it.
Let’s say, it was really important to you that your AI never spoke to you using em dashes. Sure, you can add a rule to your AGENTS.md. “Never speak to me using em dashes.” The problem is AGENTS.md is loaded once at the start of your session. So, sure for your very first response the AI will not use em dashes. However, by the time you get to your fiftieth response, it’s highly likely it will start using em dashes again.
Maybe the answer is to append your AGENTS.md to every prompt? Surely that would keep it front of mind. While that might work to an extent, every prompt would carry a lot of unnecessary baggage and wasted tokens.
Breadcrumb solves this issue and enables you to keep your AIs tightly on track. It does this by using a highly targeted rules delivery mechanism which is smart enough to deliver just the right rule at just the right time via inline rules, and/or compiled rules modules.
Emergent Rules
Of course, giving your AI just the right rules at just the right time assumes you know what rules to give it in the first place.
Let's say your AI makes the annoying mistake of pushing new work to main without asking you, and it breaks production. You would add a new rule to your AGENTS.md.
"Never push to main without my say so. Ever."
But now you have a new problem. Where does that rule live? Globally? In this repo? In every repo? Pretty soon rules are duplicated, inconsistent, and mixed with machine-specific instructions.
Breadcrumb solves this issue by noticing when you correct your AI and offering to turn that correction into a rule scoped to exactly when it should apply.
Rules are stored in a SQLite database (one row per rule). From this, Breadcrumb compiles rules modules. Each module has a one-line trigger, like “building or testing Swift code,” so the AI knows exactly when to load it.
/breadcrumb 93 rules
├── inline 7 every prompt in this scope
├── development.md 32 designing, writing, or reviewing Breadcrumb code
├── builds.md 9 building or testing Swift code
├── laravel-forge.md 8 managing Laravel Forge
├── daemon.md 7 writing or changing daemon or engine code
├── release.md 6 cutting or publishing a release
├── spark-forge.md 5 operating the build Mac
├── factory.md 5 changing a Breadcrumb file, rule, or ticket
├── website.md 4 working on innerloop.works
├── meetings.md 4 meeting capture, diarization, or naming
├── migrations.md 3 any schema or data migration
└── deploy.md 3 deploying the daemon or an app bundle
What I love about this is as rules accumulate the effect compounds. Instead of repeatedly correcting my AI each correction becomes part of how it works from then on.
My Breadcrumb currently has 93 rules. 47 of them have already been improved through actual use. I didn't sit down and write those rules. The AI created and refined them as we worked.
Emergent Workflows (auto-triggered)
I’m kind of underselling how rules end up helping. In a lot of cases they evolve into auto-triggered workflows. A good example of this is from a session at my day job Friendbuy.
I was on a Zoom meeting with a couple of colleagues. We were walking through a big feature that had gone into QA. I wasn't taking any kind of notes, just running Breadcrumb in the background. After the meeting I said to Claude:
"Review the meeting I just had, find all the bugs we discussed, and create JIRA tickets with screenshots."
It read the transcript. Found every bug we mentioned, pulled the screens from the screen timeline, spun up my local server, diagnosed each bug, reviewed the code, created 14 tickets in JIRA with diagnosis and likely fixes, attached 12 screenshots as evidence, and created a nice Slack message to send to my colleagues with the updates.
The key point is, there is no meeting-to-JIRA workflow anywhere in Breadcrumb. It emerged from a handful of rules the AI had offered me one at a time while it watched me work. Each rule kicked off the next rule which ultimately became a full workflow.
Recording Everything
When your AI can pull transcripts and screenshots from a person showing you a bug in a Zoom meeting that is very helpful. In fact, letting AI have access to every moment on your screen is helpful in an unending number of ways.
But recording "every moment on your screen" is not quite how Breadcrumb works. It goes a bit deeper than that.
It records every window on your screen. And for every window it keeps three clocks: how long you were looking at it, how long it was doing something, and how long an AI was working inside it. It knows what your AI was saying down to the second.
There are many ways all this is useful but a big one is highly detailed billing reports for client work. This is a great example of the vertical integration of this product. You can instruct your AI to create a time sheet that includes all work related to a specific client. It will know exactly how to pull it from the Breadcrumb database including meetings, related AI work, and other on-screen items that are known to be related to that client. I bill my clients this way.
But that’s just one idea.
Imagine being able to query your AI and ask it to pull up all moments related to a specific bug or feature, every time you spoke about it with your team, every bit of AI transcript related to it, every time you worked on it, every rule and note related to it.
That is the ultimate compounding. It’s how I work every day, and it’s taken my work to the next level.
Privacy
“Record everything” is a dumb idea that keeps coming back. There’s a trail of dead products and companies that have tried to do it. Huge and small companies have gotten awful press from going down this path. And yet… There must be something here if people keep circling back to this idea.
So the questions are: Is it useful? Can it be done well?
I know from my own experience it’s useful. But can it be done well?
Here’s my approach to this problem:
- Local software for just one person
- Local models on a local Mac for transcription and diarization
- Local data is encrypted with SQLCipher and a passphrase that only you hold
- Ability to exclude apps, websites, words and phrases from getting recorded
- No outbound network calls other than one daily update check (with no identifying data)
- Just download and install (no account needed, no email, no subscription, etc.)
- 100% free
How will I make money?
The personal version of Breadcrumb is free and will stay free come hell or high water. I plan to make money two ways. First, a teams product where each person's Breadcrumb still runs on their own Mac but the team shares a private memory between them. Second, licensing the Breadcrumb technology to other companies.
Give it a week
You might still think recording everything is a dumb idea. I initially thought that too. But as I experimented with it and saw how private it could be, and how much leverage it creates, I changed my mind.
Don't take my word for it, take it for a test spin. Leave it running for a week and see how the benefits start to compound in your own workflow.
I know I'm only scratching the surface of possibilities Breadcrumb can offer. If you figure out new ways to work using it please let me know.
Justin Vincent
p.s. If you want to see where this goes, follow me on X.