Reddit pain mining

A repeatable process for turning Reddit threads into validated SaaS opportunities. Distilled from Pico Pitch — an open-source multi-agent system that scrapes subreddits, scores pain signals, clusters them into themes, and emits AI-ready product docs. Pico Pitch is MIT-licensed, so the methodology here maps closely to the actual code; if you want to see a working implementation, the repo is the canonical reference.

The reason Reddit beats surveys: people complain to strangers about problems they'd never volunteer in a structured interview. You get latent intent (what they're already searching for fixes for), in their native vocabulary, time-stamped, search-indexed, and with social proof in the form of upvotes and replies. A survey tells you what people say they want. Reddit tells you what's annoying them enough to type 400 words about at 11pm.

When to use this skill

• You have a vague SaaS idea and want to know whether real people actually have the problem before you build.
• You're researching a competitor's user base and want to find the gaps their users complain about.
• You're hunting for underserved niches and need to discover problems instead of validating preconceived ones.
• You're writing a landing page or PMF survey and need the exact phrasing your prospects use.
• You're an indie hacker deciding between three project ideas and need an evidence-based tiebreaker.
• You're a PM doing discovery for a new feature and the design partner pool is too small to draw from.

When NOT to use this skill

• Enterprise B2B where the decision-maker isn't on Reddit. A CISO buying SIEM software doesn't post about it. Use sales calls, analyst reports, and LinkedIn instead.
• Regulated industries with thin or anonymised communities. Healthcare, legal, finance: Reddit gives you the patient/client side, not the buyer side. Useful for sentiment, not for sizing.
• Nascent niches with no community yet. If the subreddit has 800 subscribers and three posts a week, there's no signal to mine. Go to Discord, Twitter, or in-person conferences instead.
• Ideas where the pain is hyper-local. "Dog walkers in Portland" won't have a sub. Use Nextdoor, Facebook groups, or Craigslist gigs.
• You want quantitative TAM. Reddit is qualitative-with-frequency-counts. It is not a market sizing tool. Pair it with SimilarWeb, app store rankings, or actual competitor revenue data.

The 5-step pipeline

The whole process is a funnel: thousands of raw posts → hundreds of scored signals → dozens of themes → a handful of opportunities → one brief. Each stage compresses the firehose using a different filter.

Step 1: Niche definition

Before you open Reddit, define the niche. This is the highest-leverage step and most people skip it.

Scope it broad enough to find signal. "Indie podcast hosts" has a community. "Indie podcast hosts who use Riverside and edit in Descript" does not. Pico Pitch's default subs (SaaS, startups, Entrepreneur, SideProject) are deliberately broad — you cast wide and cluster down.

Scope it narrow enough to actually serve. "Small business owners" is not a niche, it's a census category. You can't write copy for it. Aim for a phrase a prospect would self-describe with in one breath: "Shopify store owner doing under $50k/month", "freelance video editor working with agency clients", "DTC brand running Meta ads under $100k/month spend".

Write the niche as a sentence with three slots. [Role] who [does activity] using [tools or for context]. If you can't fill all three, your niche is too vague.

Pick adjacent search territory. For each niche, list 3-5 subs where they hang out plus 3-5 subs where their problems get discussed obliquely (smallbusiness, freelance, digitalnomad). The oblique subs often have higher-quality complaints because the OP isn't performing for their tribe.

Step 2: Reddit search

Once you have subs and a niche, you need to extract posts. Pico Pitch uses PRAW and scrapes subreddit.new() — i.e. the most recent posts, not top — because top is dominated by evergreen content and karma farms. New is where today's frustration lives.

Default scrape parameters. Pico Pitch defaults: 50 posts per sub, 20 top-level comments per post, parallel across subs. That gives you ~1,000 leads per sub-cluster per run. More than enough signal; you'll throw most of it away.

Skip subreddit.hot for pain mining. Hot is optimised for engagement, which means jokes, drama, and success stories. The signal-to-noise ratio is terrible.

Use search queries surgically, not as your primary collection method. Reddit search is bad. site:reddit.com "I wish there was a tool that" on Google is consistently better. Other useful Google operators: site:reddit.com "is there an app for", site:reddit.com "tired of", site:reddit.com "anyone else hate", site:reddit.com "$" "wasted".

Handle rate limits with exponential backoff, not retries. PRAW's defaults are conservative — respect them. If you hit a 429, back off for minutes, not seconds. Reddit will shadow-ban your app ID if you hammer.

Cache aggressively. Store every raw post you pull in a local table keyed by reddit_id with an upsert. Re-running analysis on the same posts is free; re-scraping is not. Pico Pitch uses Supabase; SQLite is fine for solo work.

Set a user_agent with your username. Reddit's ToS requires this. PicoPitch:v1.0 (by /u/yourusername) is the canonical format. Generic strings get rate-limited harder.

Step 3: Pain-signal scoring

This is the heart of the methodology. Every raw post gets scored on three axes before it's allowed to influence a theme. The scoring is done by an LLM (Pico Pitch uses Gemini Flash for speed), but the rubric is the IP — the model is just executing it.

Axis 1: Is there a problem at all? Most Reddit posts aren't complaints. They're questions, announcements, jokes, success stories, or self-promo. The first filter throws out anything that doesn't surface a clear pain. Pico Pitch's prompt explicitly includes a "No clear problem" fallback so the model has permission to reject.

Axis 2: Urgency / frustration intensity. A three-level scale:

• High — desperate, urgent, ASAP, frustrated, fed up, nightmare, can't take it anymore, hate, ridiculous
• Medium — struggling, difficult, challenging, need help, looking for, trying to figure out
• Low — wondering, curious, thinking about, considering, would be nice if

The phrasing matters more than the topic. "Wondering if there's a better way to handle invoices" is a Low-urgency post even if the topic (invoicing) is a billion-dollar market.

Axis 3: Willingness-to-pay signals. Four states: Yes / Maybe / No / Unknown. The model looks for:

• Explicit dollar amounts — "I'd pay $50/month for this", "burned $15k on a developer", "my current tool costs $200 and doesn't even do X"
• Budget references — "I have budget for a solution", "willing to invest", "would expense this"
• Cost-of-problem framing — "this is costing me 10 hours a week", "we lost a client over this", "I had to refund a customer because"
• Workaround evidence — "I'm using three different tools to do this", "I built a janky Zapier flow", "I have a spreadsheet that..."

The last one is the most valuable signal in the whole skill. Someone who has already cobbled together a workaround has demonstrated willingness to pay with their time, which is a stronger signal than what they say about money. Time-rich workarounds are also where the best feature specs come from — the user has already done your product discovery.

Always extract supporting quotes. Don't let the model summarise away the user's exact words. Pico Pitch's enhanced extractor pulls 1-3 verbatim quotes per post, preserving typos and informal language. These quotes become the unfair advantage when you write the landing page — they're already proven to resonate because the user wrote them about their own pain.

Step 4: Theme clustering

Individual scored signals are noise. Themes are signal. The cluster step takes hundreds of single-post complaints and consolidates them into a small number of canonical pain themes.

Two-pass clustering. Pico Pitch does this in two passes:

1. Domain grouping. Every scored post gets a free-text problem_domain (e.g. "Project Management", "Restaurant Operations", "Customer Support AI"). Group posts by raw domain. After 500 posts you'll have ~80-150 raw domains, most with 1-2 posts each.
2. Domain consolidation. Feed the list of raw domains back to a stronger model (Pico Pitch uses Gemini Pro) and ask it to merge semantically-equivalent ones into canonical themes. "Community Management / Content Moderation", "Online Community Management", and "Online Community Moderation" all become "Automated Content & Community Moderation". After consolidation you'll typically have 15-30 themes.

Set a minimum lead threshold per theme. Pico Pitch's default is MIN_LEADS_FOR_THEME = 3. Below three independent posts saying roughly the same thing, you're chasing a single loud voice. Three is the floor; five is more comfortable; ten is a genuine signal.

Batch the consolidation prompt. If you have 200+ raw domains, single-shot consolidation will time out or hallucinate merges. Pico Pitch batches in groups of 50, then merges the batch outputs. Worth replicating.

Don't auto-merge across communities. The same complaint phrased by r/smallbusiness and r/Entrepreneur is one theme. The same complaint phrased by r/restaurateur and r/programming is probably two themes wearing the same words. Sanity-check cross-community merges by hand.

Step 5: Brief generation

The deliverable. A one-page validated opportunity brief that you can show to a co-founder, a designer, or yourself in three weeks when you've forgotten why you cared. The brief structure below is adapted from Pico Pitch's BRD generator, compressed to a single page.

The brief answers six questions in fixed order: what's the problem, who has it, how do they cope today, what would they pay for relief, what's the competitive gap, and what's the one thing nobody else has noticed. Skip a section and the brief loses its weight.

See the full template below.

The scoring rubric in detail

The combined score for a post is the tuple (urgency, willingness-to-pay, has-workaround). The five tiers, in descending order of investment-worthiness:

Treat S and A as primary signal. B is a tie-breaker between themes. C is corroboration only — useful for sizing how widespread a pain is, but don't weight it. D is filler — count it, ignore it.

Upvotes are not a proxy for pain intensity. A post with 800 upvotes is popular. The 12-upvote post with a 600-word reply chain of "this exact thing happened to me" is a stronger signal. Pico Pitch stores upvote counts as metadata but doesn't weight them in scoring, deliberately.

Number of replies matters more than upvote count. Replies are people stopping to type. That's the strongest cheap signal Reddit provides.

Sub-selection heuristics

Not all subreddits are equal. The same methodology applied to two different subs will produce wildly different brief quality. Rules of thumb:

Signal-rich sub markers.

• Vertical-specific with 10k-200k subscribers. (Big enough for volume, small enough that mods kill spam.)
• Active mod team that removes self-promo. Check /about/rules.
• Median post length over 200 words. (Short posts are usually questions; long posts are usually pain.)
• Comments outnumber posts on the front page by 5x or more. (Discussion > broadcast.)
• A weekly or pinned "what are you struggling with this week" thread. (Curated firehose — start here.)

Noise-heavy sub markers.

• 1M+ subscribers in a generic category (r/business, r/marketing). Drowned by beginners.
• Heavy meme / image-post culture. Pain doesn't fit in a meme.
• Visible drop-shipping or course-selling spam. (r/entrepreneur skews this way; mine carefully.)
• Threads dominated by OP_TITLE: long-form rant, comments: "this". No discussion = no signal extraction.
• High overlap with r/all. (You want niche conversations, not popularity contests.)

The oblique-sub trick. Some of the best pain signals live in subs adjacent to your target niche. Looking for fitness-app ideas? r/fitness is too noisy. r/xxfitness (women-specific) has higher-trust discussions. r/loseit and r/intermittentfasting surface different complaint sets. Looking for dev-tool ideas? r/ExperiencedDevs and r/cscareerquestions complain about tools differently than r/programming does.

Always check the sub's age and post velocity before mining. A new sub (<2 years) with low post velocity gives you a thin, time-skewed sample. An old sub with high velocity gives you a robust cross-section. Mine the latter.

The brief template

The one-page output. Six sections, each answers one question. Aim for a single side of A4 / Letter. If it spills, you're over-explaining.

# Opportunity Brief: [3-5 word title]
_Generated YYYY-MM-DD · N posts analysed · M themes consolidated_

## 1. Problem
One paragraph stating the pain in plain English. Lead with the most representative
verbatim quote from the scored posts. Frequency count: "Surfaced in N independent
posts across M subreddits." No abstractions, no marketing language.

## 2. Who has it
The target user as a one-sentence sketch: role, context, scale. Then 3-5 bullet
markers that disqualify lookalikes (e.g. "Has more than 3 clients" rules out
hobbyists; "Uses Shopify, not WooCommerce" narrows the integration target).

## 3. Current solutions / workarounds
What people are doing today instead of buying your thing. List the named competitors
and the duct-tape workarounds in roughly equal weight. Workarounds are where the
feature spec lives — quote them verbatim. Note any explicit price points users
mention paying for existing tools.

## 4. Willingness to pay
Direct $ signals from the scored posts: how much, for what relief, with what
ROI framing. If no explicit dollar amounts, fall back to time/cost-of-problem
framing ("losing X hours a week", "lost a client over this"). State the inferred
price band as a range, not a point estimate.

## 5. Competitive gap
What the named competitors fail at, in users' own words. This is the wedge.
Be specific: "Tool X exists but has no API" or "Tool Y exists but is desktop-only"
is a real gap. "Tool X exists but isn't great" is not.

## 6. Distinguishing insight
The one non-obvious thing you noticed that other people doing surface-level
discovery would miss. Often this is a cross-cutting workaround pattern, a
mismatch between what users say they want and what they actually do, or a
sub-segment that's louder than the headline persona. This is the section that
makes the brief worth re-reading.

---
### Evidence appendix (not part of the one-pager)
- Source posts: [permalink list]
- Scored signal distribution: S=__, A=__, B=__, C=__, D=__
- Top supporting quotes (3-5 verbatim, with permalinks)
- Pain-point frequency by sub

Keep the appendix in the file but not on the one-pager. The one-pager is the artifact you re-read; the appendix is the receipts when someone challenges you.

Anti-patterns

• Chasing single loud voices. One 2,000-word rant with 400 upvotes is one data point. Wait for three independent posts before you treat it as a theme.
• Treating upvotes as pain intensity. Upvotes measure relatability, not urgency. A post can be highly upvoted because it's funny, validating, or hits a tribal nerve — none of which mean people will pay to fix it.
• Mistaking jokes for pain. r/ProgrammerHumor and r/SmallBusiness both have heavy meme cultures. A complaint dressed as a joke is not a buying signal. The model can be fooled here; spot-check the high-urgency posts manually.
• Confirmation-biased subreddit selection. If you already think you want to build a tool for indie podcasters and you only mine r/podcasting, you'll find what you came to find. Always mine at least one adjacent sub where your hypothesis would not be obvious.
• Negative survivorship bias. Reddit over-indexes on people having problems. The 90% who quietly use the existing tool and are fine never post about it. Always sanity-check that your pain isn't "the 10% of churned users complaining loudly".
• Over-broad niches that don't convert to product. "Small business owners want better software" is true and useless. If you can't write the landing page headline from the brief, the niche is too broad — re-scope and re-mine.
• Skipping the workaround question. A pain without a workaround is often a pain people have learned to live with. Find the duct-tape, find the budget.
• Building from a single session's data. Reddit's frontpage cycles. Mine across at least two sessions a week apart and only treat themes that show up in both as durable. Pico Pitch's session-tracking exists for exactly this reason.

Worked example

Niche: freelance video editors working with agency clients. Hypothesis: there's a pain around revision management — collecting timestamped feedback from non-technical clients without losing notes across email, Frame.io, Slack, and Loom.

Step 1: Niche

Sentence: Freelance video editors who handle agency-client revisions using a mix of Frame.io, Loom, and email. Three slots filled. Subs to mine:

• Primary: r/videoediting, r/editors, r/freelance
• Oblique: r/agency, r/smallbusiness, r/AskAgency

Step 2: Scrape

Pull 50 posts per sub from .new(), 20 top-level comments each. ~1,200 raw leads. Run weekly for three weeks before relying on themes (durability check).

Targeted Google fallback: site:reddit.com/r/editors "feedback" "wish there was", site:reddit.com/r/freelance "client revisions" "tired of".

Step 3: Score (sample)

Four invented but representative posts from the corpus:

POST A — r/editors
"I have 6 active agency clients and every single one gives feedback in
a different way. One emails timestamps, one screen-records on Loom and
expects me to transcribe, one writes ¶ paragraphs in Slack. I spent
4 hours last week JUST consolidating feedback before I could edit. I'd
pay $40/mo right now for something that turned all of these into a
single threaded list against the timeline."

→ Urgency: High ("tired of", "JUST consolidating", explicit time cost)
→ Willingness-to-pay: Yes (explicit $40/mo)
→ Workaround: Yes (manual consolidation, 4 hrs/week)
→ Tier: S

POST B — r/freelance
"Frame.io is fine but my clients refuse to log in. They want to comment
in Gmail. I keep losing notes. Anyone have a flow for this?"

→ Urgency: Medium ("refuse", "keep losing")
→ Willingness-to-pay: Maybe (frustrated with existing tool but no $ stated)
→ Workaround: Yes (Gmail thread, lossy)
→ Tier: A

POST C — r/videoediting
"Would be cool if Premiere had built-in client review."

→ Urgency: Low ("would be cool")
→ Willingness-to-pay: Unknown
→ Workaround: None mentioned
→ Tier: D — noise

POST D — r/agency
"We hired 3 editors this quarter and onboarding each one's feedback
workflow is killing our PMs. We pay $200/mo for Frame.io across the
team but it doesn't solve the client-side problem at all. Would
expense a tool that fixed this in a heartbeat."

→ Urgency: High ("killing our PMs")
→ Willingness-to-pay: Yes (explicit $200 baseline + "would expense")
→ Workaround: Yes (Frame.io insufficient)
→ Tier: S (and it surfaces a B2B buyer, not just freelancer)

Step 4: Cluster

Raw domains from the full 1,200-post corpus would include: Client Feedback, Video Review Workflow, Agency-Client Communication, Revision Management, Editor PM Tools, Asynchronous Video Notes, etc.

After consolidation, the canonical theme emerges as "Multi-Channel Client Feedback Consolidation for Video", with ~14 posts across r/editors, r/freelance, r/videoediting, and r/agency. Above the MIN_LEADS_FOR_THEME=3 floor. Two adjacent themes (Client Onboarding for Creatives, Async Video Notes) sit nearby but don't merge — keep separate.

Step 5: Brief

# Opportunity Brief: Unified Video-Review Inbox
_Generated 2026-05-22 · 1,212 posts analysed · 22 themes consolidated_

## 1. Problem
Freelance video editors and small agencies waste 3-6 hours per project
consolidating client feedback that arrives across email, Loom, Slack,
and Frame.io comments. "I spent 4 hours last week JUST consolidating
feedback before I could edit" (r/editors, 87 upvotes, 43 comments).
Surfaced in 14 independent posts across 4 subreddits over a 3-week
collection window.

## 2. Who has it
Freelance video editors and 2-10 person video agencies serving non-technical
agency clients.
- Has 3+ concurrent clients (rules out hobbyists)
- Bills $2k-$15k per project (rules out wedding/event tier)
- Already pays for Frame.io or similar (proves willingness to pay for tooling)
- Clients are marketers / brand managers, not other editors (i.e. won't adopt
  editor-native tools)

## 3. Current solutions / workarounds
Named tools: Frame.io ($15-$25/user/mo, mentioned in 9/14 posts), Wipster,
Vimeo Review. All fail the same way: require client login, which clients refuse.
Workarounds in users' words:
- "I copy Loom transcripts into a Notion doc with timestamps by hand"
- "I have a Gmail filter that flags revision emails and I batch them weekly"
- "I built a Zapier flow that pulls Slack messages into Airtable. It breaks."
Average reported time tax: 3-6 hrs/project.

## 4. Willingness to pay
Explicit price points from the corpus: $40/mo (solo editor), $200/mo team
baseline (agency), "would expense" (agency PM). Inferred band: $20-$50/mo
solo, $150-$400/mo team. ROI framing consistently: hours/week recovered ×
billable rate.

## 5. Competitive gap
Frame.io and competitors are editor-side tools that require client login.
The competitive gap is **client-side asynchronous feedback capture** —
specifically: ingesting feedback from the channels clients already use
(email, Loom, Slack) and presenting it to the editor in a single timeline-
anchored view. No named competitor does this; all 14 posts mention at
least one existing tool as inadequate.

## 6. Distinguishing insight
The buyer is not always the user. In 6 of 14 posts, the person complaining
is an agency PM or producer, not the editor themselves — they want to
*stop their editors complaining*. This suggests two distinct ICPs (solo
freelancer self-buy + agency PM team buy) with the same product but
different pricing/positioning. Most surface-level discovery on this niche
treats it as a single solo-editor market. The agency PM segment is louder
and has bigger budgets.

---
### Evidence appendix
Source posts: [14 permalinks]
Scored signal distribution: S=4, A=6, B=2, C=2, D=0
Top quotes: [3 verbatim with permalinks]
Pain-point frequency: r/editors=6, r/freelance=4, r/videoediting=2, r/agency=2

This brief took ~30 minutes of analysis time on a corpus that took ~10 minutes to scrape. The Step 6 insight (two ICPs, PM segment is louder) is the kind of finding that doesn't fall out of a survey — it requires reading actual posts in actual subs.

Further reading

• Pico Pitch on GitHub — the open-source multi-agent implementation this skill distills. The prompts/ directory contains the production scoring and clustering prompts; reddit_scraper_agent.py shows the PRAW collection patterns; orchestrator.py shows the 6-stage pipeline end-to-end.
• The Mom Test by Rob Fitzpatrick — the canonical book on why people lie in surveys and how to ask validation questions that surface real behaviour. Reddit pain mining is the asynchronous version of the Mom Test.
• PRAW documentation — the Python Reddit API wrapper. Read the rate-limit section before you write a single line.
• Reddit API terms — the rules. Set your user agent, respect 429s, don't store deleted content.
• GummySearch — paid SaaS that does a lighter version of this workflow with a UI. Useful as a sanity check against your own pipeline; also useful as a competitive reference for what not to build.
• F5Bot — free keyword-alert tool for Reddit. Set it on your competitors' brand names and your niche's pain phrases to keep the signal coming after the initial mine.
• "How to find your first 10 customers on Reddit" discussions on Indie Hackers — community-curated tactics that overlap with this skill but lean more toward outreach than discovery.