55 tests. 6 principles. 1 tool the team actually used.
How I turned a year of Care.com's experiment data into six working principles and an internal tool — and how the same thinking, built independently, became my product Content Playbook Engine.
A year of answers nobody could find
Care.com's growth team ran content experiments constantly. Headlines, CTAs, pricing copy, reassurance messaging — tested, measured, logged in a tracker, and then mostly forgotten.
So every new project started from zero. A designer writing a new upgrade module had no easy way to know we'd already tested that exact idea — twice. The evidence existed. It just didn't travel.
The tool's first job was to read that pile and find the signal. Fed a year of raw results, it isolated the content tests — the ones that changed the words users read, not the layout or the pricing — and synthesized them into six plain-language principles, each traced back to the specific tests that earned it. Not my six principles. The evidence's six principles, surfaced by the tool and grounded so nobody had to take anyone's word for them. Those principles started informing copy and UX decisions across the product.
A document gets read once. A tool gets used.
Principles in a doc have a shelf life. I wanted the evidence to live where decisions happen — searchable, browsable, mapped to the product itself.
So I built the app. Working with AI as my build partner, I turned the synthesis into an internal tool the product team could open any morning:
- Browse by principle. Every best practice, with the tests behind it and how strong the evidence is.
- Browse by funnel stage. The enrollment journey laid out end to end, showing where users responded well, where they resisted, and where we'd never tested at all.
- Submit a test. PMs and designers could add new results themselves, so the playbook grew instead of aging.
The point was never the app. The point was that a designer starting a new project could answer "what do we already know about writing for this screen?" in thirty seconds.
The hard part wasn't using AI. It was knowing where to stop it.
Anyone can feed data to a model and get confident-sounding output. The real design work was deciding what the AI should never be allowed to do. Four rules made the tool trustworthy:
Code makes the calls. The model writes the prose.
Anything a stakeholder reads as a judgment — did this test win or lose, is this stage healthy — is computed from the numbers, in code, the same way every time. If the model re-decided those calls on every run, the same data could tell a different story on Tuesday than it did on Monday. That's not a tool; that's a mood.
Report behavior, never attitude.
The tool says what users did — clicked, converted, dropped off. It never claims what they felt. "Shorter copy won here" is evidence. "Users have commitment anxiety" is a guess wearing a lab coat. Humans can make that inference; the tool doesn't make it for them.
Losses are findings.
A stage where three variants failed in a row isn't a gap in the playbook — it's one of its most useful pages. "Stop adding urgency messaging here" saves as much money as any winning headline.
Show what you can't resolve.
When two tests genuinely contradicted each other, the tool named the conflict instead of averaging it into a fake answer. A playbook that only ever produces tidy wins is lying, and experienced PMs can smell it. The honesty is what makes the rest believable.
These four rules are content design. Not the copy on the buttons — the rules governing what the system is allowed to say. That's what writing for AI actually means.
See Content Playbook Engine as a consulting engagement →