Reading time:

11 min read

|

Last updated:

The Curse of Knowledge in Product Design

The curse of knowledge in product design: why founders can't judge their own first run, and the cold-read tests that show where new users stall.

We design websites and products that make AI companies more money.

Siddarth Ponangi

Founder, Studio Maydit

We design websites and products that make AI companies more money.

Web and product design for AI companies

We help AI companies build fast, clean, and conversion-focused websites and products.

The curse of knowledge in product design is what happens when the people who built a product can no longer imagine not knowing how it works. Every label, field and step looks obvious to them, because they already know what it means. The person who built the product is the one person who cannot judge its first run, so a founder clicking through his own onboarding tells him almost nothing.

For an early-stage AI founder this matters more than any visual choice on the screen. Self-serve products live or die in the first session, and that session is the one place where the founder's expertise works against him. You know your product and nobody else does. The fix is not more effort from you. It is getting a stranger in front of it and staying quiet while they use it.

The onboarding that takes its builder 40 seconds

Picture an AI tool that crawls a company's marketing site and writes content briefs for the pages it should publish next. The buyer is a content marketer, not an engineer. After signup there are four setup screens, each with a progress bar at the top that reads Step 1 of 4.

Screen one is titled Add your property. It has one field, Root domain, with the placeholder example.com. Screen two is Crawl settings. It has a field labelled Crawl scope (regex), prefilled with a pattern that starts with a caret, a Max depth dropdown set to 3, and two toggles, Respect robots.txt and Render JavaScript. Screen three asks you to connect Search Console, with a grey Skip for now link under the blue button. Screen four is Seed entities, a tag input with the helper text Add 3 to 10 entities to anchor topical clusters. The last button says Start crawl.

The founder runs this flow on staging before every release. He types his own domain, tabs straight past the regex field, skips Search Console, types three entities from memory and hits Start crawl. It takes him about 40 seconds and nothing ever breaks, so it passes.

Now picture a content marketer on a recorded call, someone who has never seen the product. She pastes her blog URL, with https and a trailing slash, into Root domain and gets a red line reading Enter a root domain without protocol. She fixes it on the second try. On Crawl settings she stops. She hovers the small info icon next to the regex field and reads Use RE2 syntax. Leave blank to crawl all paths under the property. She does not know if blank is safe, or if the prefilled pattern will crawl her whole site, or only part of it. She deletes it, then puts it back. Six minutes in, she is still on screen two. That is where a real signup closes the tab.

Why the regex field looks fine to the person who wrote it

The founder is not careless. He wrote the crawler. When he sees Crawl scope (regex) he sees the decision behind it: some customers have huge sites, crawls cost money, so the scope needs a limit. The label is a short note to himself about a choice he already made. To him it reads as complete, because his own memory fills in everything the screen leaves out.

Psychologists call this the curse of knowledge, and one experiment shows it clearly. In a Stanford study by Elizabeth Newton, one group tapped out the rhythm of well known songs on a table and another group tried to name them. The tappers predicted listeners would get about 50 percent of the songs. Listeners got about 3 percent. The tappers could hear the melody in their heads while they tapped. The listeners only heard knocking.

Your onboarding is a tapped song. You hear the melody: why the property comes first, what an entity is, why Search Console makes the briefs better. Your user hears four screens of knocking. And the gap does not shrink with effort. Trying harder to see it fresh is like a tapper trying harder to not hear the tune.

Dogfooding the first run can only ever pass

Most product advice tells founders to use their own product every day. For workflows and bugs, that is good advice. You will find the export that drops a column and the button that does nothing on Safari. For the first run, it is worse than useless, and here is the part most advice gets wrong.

A test is only worth running if it can fail. When the founder walks his own onboarding, it cannot fail on comprehension, because he already understands every word on it. So it returns a pass every time. That pass then gets written down. The Linear ticket says QA onboarding, the founder ticks it, and the team stops looking at the first run because someone senior has signed off on it.

The same is true of everyone near the product. The engineers built it. The first hire sat through the founder's demo in week one. The friendly beta users are often founders he knows, who were walked through it on a call. The investors saw it in a pitch with him narrating. Every person whose opinion he hears has already been told the melody.

So the founder is left reading a metric he cannot explain. Signups look healthy. The count of people who start a crawl is much lower. He reads the gap as a traffic quality problem and tightens the ad targeting, because the onboarding passed QA last Friday.

The words that only make sense from inside

Most first-run trouble is vocabulary. Words that are precise inside the company are noise outside it. In the example above, four of them sit in the first two minutes: property, crawl scope, render JavaScript and seed entities. Each one is correct. None of them is a word a content marketer uses about her own job.

You can find yours without a stranger, using material you already have. Read the last twenty support tickets and the questions prospects asked on sales calls, and mark every product word they repeat back with a question. Then open each setup screen and circle every noun the user has not been told about by that point. Those circles are your list.

For each word, choose one of three moves. Rename it into the user's words, so Seed entities becomes Topics you want to rank for. Explain it inline with an example, not a definition, so the crawl field says Leave this alone to crawl your whole site. Or move it out of the first run entirely, into settings, where only the people who need it will look. The regex field almost always belongs there. Deciding which step comes first is its own problem, and it is worth reading about why fixing UX often starts with sequencing, not screens.

Cold-read tests a founder can run alone this week

None of these need a research budget. They need a fresh account, a calendar invite and the discipline not to talk.

  • Make a new account with an email that has never touched the product, in a private window with no saved logins. Your own account skips steps that a new user sees.

  • Recruit two or three people who match the buyer and have never seen the product or your demo. A friend of a customer is fine. A friend of yours who heard the pitch at dinner is not.

  • Send them the signup link and nothing else. No context, no deck, no Loom. A real signup arrives with only your website in their head.

  • Ask them to think out loud and record the call. Then mute yourself. Every time you explain something, you have fixed the problem for one person and hidden it from yourself.

  • Write down the exact second they stop moving and the exact words on screen at that moment. That screen and those words are your finding.

  • Stop the session when they reach the first useful output, or after ten minutes. If ten minutes passes before the first result, that is the result.

  • Do not wait for a big sample. Nielsen Norman Group found that five users reveal about 85 percent of usability problems, and a single user shows you almost a third of them. Two sessions this week beat twenty next quarter.

Count what you would never count on your own run: time to the first result, number of fields edited and then reverted, and every place they hovered a help icon. Those three numbers are the honest version of the 40 seconds.

Who should watch the first session

The founder should watch, but he should not be the only one, and he should not be the one who decides what it means. He will want to explain each stop away. She just did not read the tooltip. She is not really our buyer. That is the curse speaking again.

Put the engineer who built the crawl settings screen in the room. Engineers rarely get to see a stranger use their work, and watching someone hesitate over a field they shipped changes how they write the next one faster than any review comment. Add whoever writes your product copy.

Agree on one rule before the call: nobody speaks to the participant except to say keep going. After the call, everyone writes down the screen where it stalled before discussing it. If three people independently name screen two, there is nothing left to argue about. If the stall is happening later, after setup, our step by step method to fix SaaS onboarding drop-offs shows how to count users at each step and find the cliff.

What it costs to keep grading your own first run

The signup was already paid for. Ad spend, content, the founder's evenings on LinkedIn: the cost lands at signup whether or not anyone gets past the regex field. Every person who leaves on screen two is acquisition money with nothing to show for it.

Benchmarks measured on product-led SaaS companies put activation at 20 to 40 percent for most products and 40 to 60 for the best. On those same companies, a ten point lift in activation typically drives a 15 to 25 percent increase in free-to-paid conversion. A first run that the team cannot see is often the cheapest ten points available.

For an AI product the waste is sharper. AI product builders average around 52 percent gross margin, against 70 to 80 percent for traditional software, because inference sits in cost of goods. A confused user who starts a crawl, gets lost and leaves still cost you compute. You absorb the same funnel waste with less room to absorb it. The broader patterns for getting people to value are in our SaaS onboarding design guide, but none of them help until someone outside the building has used your first run.

Studio Maydit is the stranger you can bring in on purpose. We are a product and web design studio for AI founders in the US, UK and Europe, and our product design work starts in the first run, the screens where a new user decides whether to stay. Fixed-scope projects take three to four weeks and end with a diagnosis of what is leaking in the product, written from the outside. If your onboarding passes every time you test it and your numbers say otherwise, book a 30 minute call and we will walk your first run cold.

Frequently Asked Questions

Table of Contents
Scroll to view headings
0%