Last quarter, I did something stupid. I fed ChatGPT 50 customer interviews, asked it to “find the patterns,” and built features based on what it told me.
The AI was confident. The summaries were clean. The recommendations seemed logical.
None of it worked.
Adoption rate? 14%. Customer feedback? “This doesn’t solve my actual problem.” Engineering team morale? Let’s not talk about it.
Here’s what I learned the expensive way: AI is really good at telling you what most people say. It’s terrible at finding what matters.
Three months ago, I was reviewing user research for a new workflow feature. I had:
I threw it all into GPT-4, asked for insights, and got back a beautiful summary. It told me users wanted “better collaboration features” and “improved notifications.”
So we built better collaboration features and improved notifications.
Launch day: crickets.
Then I did something I should’ve done first. I actually read the transcripts myself. All of them. Took me two full days.
Buried in interview #37, a user mentioned something casually: “I usually just screenshot the data and text it to my team because your export feature makes the file too big for email.”
That throwaway comment appeared once. In one interview. Out of fifty.