Product Metrics That Actually Matter: Beyond Vanity KPIs
A practical guide to choosing and tracking metrics that drive real product decisions, with examples from B2B SaaS, consumer apps, and marketplace products.
Product Metrics That Actually Matter: Beyond Vanity KPIs
Every product dashboard I've seen contains at least three useless metrics. You know the ones:
- Total registered users (includes spam and inactive accounts)
- Page views (counting my own QA testing)
- "Engagement" (undefined)
- Know in 30+ days
- Can't fix quickly
- Know in 1 day
- Directly improvable through AI tuning
- Definition: % of signups who reach "aha moment" within 7 days
- Why it matters: Predicts all downstream metrics
- Target: 40%+ for PLG (Product-Led Growth), 60%+ for sales-led
- Activation = Merchant installs app + gets first recommendation shown to customer
- We tracked: Time to first recommendation (target: <15 minutes)
- Lever: Simplified onboarding from 8 steps to 3
- Definition: % of active users who use the core value prop
- Why it matters: Differentiates engaged users from tire-kickers
- Target: 60%+ for core features
- Core feature: AI-scheduled events
- Metric: % of weekly active users who accept at least 1 AI suggestion
- Target: 70% (anything less means we're not solving the problem)
- Definition: % of users still active after N days/weeks/months
- Why it matters: Growth is impossible without retention
- Target: Day 30 > 20%, Month 6 > 40%
These are vanity metrics—they go up and to the right, but they don't help you make better product decisions.
After years of building products across B2B SaaS, consumer apps, and e-commerce, I've learned: The right metrics are boring, specific, and actionable. Here's how to find them.
The Metric Selection Framework
Rule 1: Metrics Must Drive Decisions
Bad Metric: Total user signups Why: Doesn't tell you if product is healthy or what to do next
Better Metric: Week 1 retention rate by acquisition channel Why: Directly actionable—double down on high-retention channels, fix or abandon low-retention ones
The Test: If a metric changes 20%, what action would you take? If the answer is "nothing" or "investigate more," it's not actionable.
Rule 2: Metrics Must Be Leading, Not Lagging
Lagging Indicators: Tell you what happened (revenue, churn) Leading Indicators: Predict what will happen (activation rate, feature adoption)
Example from AskShop.ai:
Lagging: Merchant Monthly Recurring Revenue (MRR)
Leading: Recommendation Relevance Score
The Pattern: Leading indicators let you course-correct; lagging indicators confirm you're already off course.
Rule 3: Metrics Must Have Clear Ownership
Bad Setup: "Growth team owns user growth" Problem: Too vague—what specific levers do they control?
Good Setup: "Growth team owns activation rate, defined as users who complete onboarding and use core feature within 7 days" Why: Crystal clear what to measure, what to improve, and who's responsible
The Core Metric Framework by Product Type
B2B SaaS Products
The Hierarchy:
1. Activation Rate (Most Important)
Example—AskShop.ai:
2. Feature Adoption Rate
Example—Calendare:
3. Retention Cohorts
The Dashboard:
Cohort | Day 1 | Day 7 | Day 30 | Month 3 | Month 6
----------|-------|-------|--------|---------|--------
Jan 2025 | 100% | 45% | 22% | 15% | 12%
Feb 2025 | 100% | 52% | 28% | 18% | -
Mar 2025 | 100% | 58% | 31% | - | -
Reading Cohorts:
4. Net Revenue Retention (NRR)
The Math:
Starting MRR: $100K
Expansion: $25K (customers upgraded)
Churn: $10K (customers cancelled)
NRR = ($100K + $25K - $10K) / $100K = 115%
What This Means: Even with zero new customers, revenue grows 15% annually.
5. Time to Value (TTV)
Consumer Apps
The Hierarchy:
1. Daily Active Users / Monthly Active Users (DAU/MAU)
Benchmarks:
2. Session Frequency
3. Session Duration
The Trap: Longer isn't always better
4. Viral Coefficient (k-factor)
The Math:
100 users invite 5 friends each = 500 invites
20% accept = 100 new users
k-factor = 100 new / 100 existing = 1.0
5. Retention Curves
The Shape:
Perfect: ——————— (flat from day 1)
Great: ————\_____ (stabilizes quickly)
Good: ————\\____ (stabilizes eventually)
Bad: ————\\\\\\\ (never flattens)
Marketplace / Platform Products
The Hierarchy:
1. Liquidity Score
Example—Ride-sharing:
2. Take Rate
Examples:
3. Buyer/Seller Balance
Example—Freelance Marketplace:
4. Repeat Transaction Rate
5. Multi-Tenanting Rate
Example—Food Delivery:
The Anti-Patterns: Metrics That Mislead
Anti-Pattern 1: Counting Everything
The Trap: Track 50+ metrics, none actionable
Example Dashboard:
The Problem: Can't see the forest for the trees
The Fix: Hierarchical metrics
Anti-Pattern 2: Benchmarking Without Context
The Trap: "Our conversion rate is 2%, industry average is 5%, so we're failing"
The Problem: Context matters more than absolute numbers
Example:
The Fix: Benchmark against yourself over time, not against others
Anti-Pattern 3: Optimizing Sub-Metrics
The Trap: Improve a metric that doesn't matter
Example from HerbalBath:
The Fix: Always connect metrics to revenue/retention/growth
Anti-Pattern 4: Ignoring Segmentation
The Trap: Averages hide the truth
Example:
The Fix: Always segment
The Practical Implementation Guide
Step 1: Define Your North Star Metric
The Question: If you could only track one metric, what would it be?
Good North Star Metrics:
Bad North Star Metrics:
The Test: Does this metric: 1. Measure value delivered to customers? ✓ 2. Predict business outcomes? ✓ 3. Guide product decisions? ✓
Step 2: Map Input Metrics
The Question: What drives your North Star?
Example—Slack (North Star: Messages Sent):
Input Metrics: 1. Team activation rate (% of teams with >3 members active) 2. Channel creation rate (more channels = more messages) 3. Integration usage (tools connected = more use cases) 4. Mobile adoption (access anywhere = more messages)
The Relationship:
More integrations → More use cases → More channels → More messages
Step 3: Set Up the Dashboard
The Structure:
Top Section: North Star
Middle Section: Input Metrics
Bottom Section: Segmentation
Tools:
Step 4: Weekly Review Ritual
Monday Morning (15 minutes): 1. Check North Star trend 2. Identify biggest mover (up or down) 3. Dig into that metric's segment data 4. Form hypothesis about cause 5. Decide: investigate further or ship experiment
Friday Afternoon (30 minutes): 1. Review experiments launched this week 2. Check if input metrics moved as expected 3. Plan next week's priorities based on data
Monthly (2 hours): 1. Deep dive on one key metric 2. Cohort analysis 3. User interviews with high/low usage segments 4. Strategic planning based on learnings
Real Examples: Metrics in Action
Example 1: Improving Activation at Calendare
Problem: Only 15% of signups complete onboarding
Investigation:
Hypothesis: Permission request is scary
Experiment:
Result:
The Lesson: One metric (activation) led to clear experiment, drove downstream impact
Example 2: Fighting Churn at AskShop.ai
Problem: 25% of merchants churned after first month
Investigation:
Hypothesis: Merchants whose customers don't see recommendations think it's not working
Experiment:
Result:
Action: Changed onboarding to focus on widget placement
The Lesson: Found the leading indicator (early recommendations) that predicted lagging indicator (churn)
Example 3: Growing HerbalBath Revenue
Problem: Traffic growing, revenue flat
Investigation:
Hypothesis: Single-purchase customers are the problem
Experiment: Subscription offering
Result:
The Lesson: Focused on repeat purchase rate (input metric) instead of conversion rate (vanity metric)
Conclusion: Metrics as a Product
Your metrics are a product—for your team. They should be:
1. Simple: Anyone can understand them 2. Accessible: Everyone can see them 3. Actionable: They drive decisions 4. Aligned: Everyone optimizes for the same things
The Anti-Pattern: Metrics as surveillance (tracking to catch people, not to improve product)
The Goal: Metrics as enablement (helping teams make better decisions faster)
Final Checklist:
For each metric you track, ask:
If you answer "no" to any question, remove the metric.
Better to track 5 metrics well than 50 metrics poorly.
---
Further Resources:
Tools to Try: