DateBot
A personalized AI decision engine for dating.
- Role
- Product Engineer
- Year
- 2026
- Platform
- iOS
- Status
- On the App Store
- Stack
- Swift · SwiftUI · TypeScript · PostgreSQL · Redis
Drag for more · click a screen to open it
DateBot is a dating decision system, not a text generator.
The hard part wasn't getting a model to write a decent reply. It was getting the system to understand who the user was talking about, what had already happened, whether there was actually a decision to make, and what the next move should be before generating any wording.
I ended up separating the system into evidence extraction, context management, structured decision making, response generation, and asynchronous memory updates.
01Architecture
- 01Multimodal inputtext · screenshot
- 02Evidenceextraction
- 03Context assemblyconversation · person context
- 04Structured judgmentsufficiency · action
- 05Responsemessage · optional reply
- 06Asynchronous consolidationafter the response
- 07Durable person contextprojection
- 03Context assemblynext turn
The important part is where the boundaries are.
- conversation history is canonical
- context assembly is deterministic and replayable
- semantic judgment happens before wording
- fresh explicit evidence outranks stale durable memory
- durable memory updates happen asynchronously
- identity is never silently merged
02From generation to judgment
I didn't want a model to generate prose and then have the rest of the system infer what decision it had made.
The reasoning step returns a structured judgment first. The wording is produced from it.
The contract separates information sufficiency, action selection, and response realization. That stops the writing layer changing the underlying decision, and makes restraint, including waiting or doing nothing, a first-class outcome rather than a failure to produce text.
Response generation cannot change the selected move.
The behavioral policy inside the contract is omitted.
03How the intelligence architecture evolved
The architecture did not start where it ended.
Model candidates were evaluated independently for the reasoning path and for supporting workloads, rather than treating a newer model as universally better. A substitution was a controlled architecture change: one variable, requalified before promotion.
A production trace later revealed that a secondary generation stage was being invoked outside its intended responsibility, on turns that did not need it. I stopped the tuning work, audited the runtime path before changing anything, and restored the ownership boundary rather than compensating with another prompt.
The corrected ownership boundary is the decided architecture. A fresh post-fix production trace has not yet re-verified it in the shipped binary.
04Built, measured, removed
Some of the best architecture decisions were deletions. I built these, measured them, and removed them when the evidence showed they duplicated work or put semantic reasoning in the wrong layer.
- a pre-judgment extraction pass
- became a second semantic reasoning system on the blocking path
- a separate ask / answer gate
- duplicated a decision the contract already owned
- per-message memory learning
- forced a derived projection to behave like live state
- an unconditional second-stage renderer
- added another model pass without fixing the underlying decision
- runtime semantic-English validators
- pushed language interpretation back into deterministic code
The clearest one was deleting something after proving it worked. A subsystem had been production-verified, and a later comparison showed a stronger model could improve it materially. I removed it anyway: the real problem was architectural, not quality.
Better model. Wrong abstraction.
The same logic decided the asynchronous memory path. In a frozen candidate comparison, one candidate produced a catastrophic wrong-person durable write on an explicit person switch. The selected model did not, and recall was otherwise comparable.
For durable memory, attribution safety mattered more than squeezing out another point of recall.
05Longitudinal memory
The memory problem ended up being much harder than "save the chat history."
Conversations are containers. People are not.
The same person shows up across different chats, weeks apart, with new information in between. New evidence has to be useful immediately, and response latency cannot wait on durable memory being compiled.
So the transcript stays canonical and durable person context is a derived projection rebuilt from it, never the source of truth. Fresh evidence is available to the next response while consolidation runs asynchronously.
- Conversation history+ recent evidence
- Context assemblydeterministic
- Completed interaction
- Asynchronous consolidation
- Durable person context
Implementation details for memory reconciliation, identity safety, and commit semantics are intentionally omitted.
1906 / 1906 deterministic tests · 20-turn, 6-conversation staging soak · 44 jobs, 0 failed
06Evaluation infrastructure
Evaluating DateBot turned out to be harder than building the first version. LLM output moves around when nothing changes, and I was also the person deciding what "better" meant, which makes it easy to convince yourself a change worked when you got a lucky sample.
So every meaningful change ran against frozen exact-input replays with repeated sampling, held-out cases, known-good anchors, and one controlled variable. Provenance was read from the runtime rather than trusted from documentation.
- frozen exact-input replay
- held-out cases cannot be the cases that motivated the change
- known-good anchors run beside failure cases
- exactly one causal variable changes per experiment
- blind, order-randomized review when taste is the instrument
- claims are withdrawn when replication fails
A needed fact missing from the frozen context is an input failure, not a reasoning failure. A good-sounding answer does not get to hide the distinction.
input · context · judgment · contract · realization · taste
- 0 / 74
- catastrophic failures at requalification across 25 cases
- 14 / 15
- cold-start sufficiency with zero unnecessary questions
- 6 · 5 · 0
- improved · preserved · degraded across 11 held-out situations and 3 held-out people
Requalification ran with durable memory disabled; cold-start sufficiency on a sparse window. These are defined evaluation sets, not general accuracy claims. Sample counts, thresholds and gate composition are omitted.
07When the evidence said I was wrong
A single-sample result looked like a real mechanism. It did not survive replication.
- 1 / 3 vs 3 / 3
- initial reading, one sample per arm
- 4 / 5 vs 5 / 5
- replication, Fisher exact p = 1.0
The apparent mechanism disappeared under replication, so I withdrew the claim. The corrected record showed the supposedly failing behavior succeeding in four of five samples.
Separately, a launch-day dogfood session produced nine complaint classes and one obvious-looking suspect. I traced all 62 production turns from that day, failures and positive anchors alike. Every one had run on the same byte-identical runtime bundle, so there was no release regression.
The failures split across person scoping, judgment framing, realization variance, and an under-specified handoff. No prompt change was made until the failure owner was established.
08The most consequential eval bug was provenance
For several days the architecture documentation named one reasoning model while the runtime was serving another. One campaign was preregistered against the documented model and executed on the deployed one.
That invalidated a regression story I thought I had: the baseline I was comparing against had never run in production. After that, every eval artifact recorded the resolved model identity from the runtime, and the harness asserted it before spending anything.
Never trust the prose label.
09Qualifying what ships
A model is never qualified by itself. Production behavior is qualified as a versioned bundle, and a candidate change is a substitution into it rather than an edit of it.
Unsupported configurations fail closed at boot rather than running in a state nobody qualified. The previous bundle stays the rollback baseline, and I do not mutate the current one to accommodate a candidate.