Echolab

Overview

Echolab is an AI-powered product intelligence tool I led from concept to launch. It reads unstructured customer feedback (support tickets, app reviews, chat logs) and turns it into prioritized, testable A/B experiment ideas grounded in credible industry evidence.

Each idea follows one simple format:

If we [change], then [KPI] will improve because users say [pain point].

Every hypothesis comes with real customer quotes as proof, an impact and feasibility score, and a one-click push into Jira, Optimizely or GrowthBook.

The problem

Product teams receive thousands of customer messages a month, but the useful insights get buried. We found three pain points:

• Insights lost in the noise. Thousands of messages go unread.
• Slow experiment cycles. Manual tagging and spreadsheet exports meant 3–4 week analysis cycles.
• A trust gap in AI suggestions. PMs didn’t trust AI ideas that weren’t backed by evidence.

Voice of the customer

I ran discovery sessions with PMs across SaaS, energy, education and health tech.

“Hours of triage was my #1 pain point. Ideas were essentially dropped on the floor because there was no time to sort through and group common requests and bugs.” — AI PM

“Our users are healthcare professionals who are not patient when tools lag. If AI could cluster similar complaints and show me where patient safety might be at risk, we could act fast.” — PM, HealthTech

The pattern was clear: high volume, many personas, and no time. Usability, security and privacy were non-negotiable.

Discovery and validation

• Interviews: PMs, support leads and UX researchers all knew insights were buried but lacked time to extract them.
• Sizing: a median company handles 500–5,000 feedback items a month.
• Competitive scan: Medallia and Qualtrics existed but were heavy, expensive and not built for hypothesis-driven iteration. The gap was a lightweight, AI-first workflow for PMs.

AI hypothesis

If we automate the classification, clustering and grounding of customer feedback in proven solution patterns, product teams can launch experiments faster and with more confidence, because the ideas will be evidence-backed and directly actionable.

Scope: what we do and don’t do

We generate A/B ideas for feature-improvement complaints like confusing workflows, onboarding drop-off, poor feature discovery or slow perceived performance. These have more than one possible fix, so testing makes sense.

We do not handle clear-cut bugs, outages or compliance issues. If there’s one obvious defect, you fix it rather than test it.

Personas

• Primary: Product Managers want less backlog review time and trusted, prioritized experiments.
• Secondary: UX Researchers and Analysts want to put numbers on qualitative feedback.

Model selection

I ran a proof of concept comparing several embedding models and LLMs on fit for purpose, accuracy, speed and cost.

• Embeddings: we chose all-MiniLM-L6-v2 for its speed and small footprint, which made large-scale clustering and search practical.
• LLM: GPT-4o handled summarizing, classifying and drafting hypotheses.
• RAG: before writing a hypothesis, the system pulls relevant industry benchmarks, UX patterns and case studies from a curated knowledge base. This keeps ideas grounded and gives each one a citable source.

Data pipeline

1. Collect feedback from Zendesk, Intercom and app stores.
2. Clean it: remove duplicates and greetings, mask personal information (GDPR/CCPA), tag sentiment, classify bug vs. improvement.
3. Store knowledge-base embeddings in Supabase (pgvector) and structured outputs in PostgreSQL.
4. Retrieve supporting evidence through the RAG layer.
5. Output structured hypothesis cards with suggested test variants.

How we validated the AI

• Spot-checked embeddings to confirm similar feedback clustered together.
• Checked RAG sources for relevance and correct attribution, with no made-up references.
• Compared hypotheses against the original raw feedback.
• Fed in data from past successful experiments to see if the model reached similar ideas.
• Measured real A/B test uplift as the final proof.

Acceptance criteria

• Hypotheses generated within 2 minutes of feedback arriving.
• Over 90% accuracy classifying bugs vs. feature requests.
• At least 3 credible external references per improvement hypothesis.
• Every hypothesis card shows source quotes and ticket IDs.

Business model

A B2B subscription with tiered pricing based on data volume and integrations, plus custom enterprise licensing.

MVP demo

Watch the MVP demo

Results

• Reduced backlog review time by 85%.
• Increased experiment launch frequency by 3x.
• Raised PM confidence in AI-driven insights by 40% in internal surveys.

What’s next

• 0–3 months: better ingestion, richer sentiment analysis, faster review UI.
• 3–9 months: more feedback sources, predictive analytics, multi-language support.
• 9–18 months: a self-learning loop where experiment results improve future ideas.