Review Analysis Methodology

Marketplace reviews are typically 85-95% positive. Random sampling misses most problems. Sellyze collects hundreds of reviews per product, up to 2,000 on Pro plans, and applies stratified sampling. "Stratified" means the sample is split into layers by star rating instead of pulled at random. Lower-rated reviews are oversampled at a higher rate than their natural proportion, while positive reviews are capped. The oversampling rate is calculated dynamically based on the actual rating distribution of each product. This produces a balanced dataset with enough negative signal to detect even uncommon issues. A product with 5,000 reviews and a 4.3 average might have only 400 negative reviews. Stratified sampling ensures those 400 are well-represented in the analysis instead of being drowned out by the 4,600 positive ones.

Each review is decomposed into individual claims using purpose-built AI models. A single review often contains multiple distinct complaints. "The lid leaks and it's hard to clean" becomes two separate claims. "Great product but the handle broke after a week and the color is not what I expected" becomes three. Claims are classified into six core categories: product_defect (something broke or failed), missing_feature (something buyers expected but the product lacks), usability_issue (the product works but is frustrating to use), preference_mismatch (subjective taste, like color or size), logistics_complaint (shipping, packaging, or fulfillment problems), and user_error (the buyer misused the product). Etsy and TikTok Shop analyses add platform-specific extensions such as sizing_fit, description_mismatch, seller_communication, and safety_concern. Claims are clustered into named pain points. Two claims are grouped if fixing one would fix the other. Models are configured for deterministic, reproducible results.

Because lower-rated reviews are oversampled, raw frequency counts overstate how common each problem is. Sellyze corrects for this using statistical weighting methods borrowed from survey research. Each review is assigned a weight based on its star rating and the actual rating distribution of the product. Mention rates are then calculated using these weights, producing prevalence estimates that reflect the true proportion of buyers affected. This step is what separates Sellyze from tools that just count keywords. Without prevalence correction, every pain point looks like a bigger deal than it actually is.

Individual product analysis has inherent sample-size limitations. The Opportunity Scanner scans up to 60 products in a category and runs deep AI analysis on the top competitors. This is where single-product findings get stress-tested. A pain point found independently in 4 out of 10 products is a category-level gap, not a single-product fluke. If buyers of different brands, at different price points, all complain about the same thing, that complaint represents a real market opportunity. A pain point that only appears in one product might just be a bad batch or a single manufacturer cutting corners. Cross-product replication is the gold standard of validation, the same principle used in scientific research.