How we count
Every number on this site is a count of reviews. Most come from the two app stores; you can also post your own on any app page. This page says where the reviews come from, which ones we leave out, and what the counts on an app page do and do not tell you.
The five steps
- 1
We pull the reviews from the two app stores
Google Play and the US App Store, in English. We take them in date order, newest first, and keep pulling until the store stops giving us more. For smaller apps, that's every review the store has. For the very large apps the store stops well before the end, so we keep only the most recent ones. Each app page shows which months those reviews cover.
- 2
We drop the ones with no words, and the ones not written in English
Reviews with no text at all, and reviews that are only an emoji or a punctuation mark, are set aside. So is the odd one that repeats the same short line over and over. Everything else stays, including the very short ones — "Can't even register" is three words and it's still someone telling you something. We ask the stores for the US English listing, but people write in whatever language they like, so reviews come back in Spanish, Russian, Arabic, German and more. We set those aside too. A review that quotes a word or two of another language stays. The language check isn't perfect, so a few non-English reviews slip through.
- 3
We sort the rest into topics
A language model reads each review and tags what it's about: fake profiles and scams, meeting someone, not meeting anyone, billing, canceling, support, the app itself. One review can carry more than one tag. This is the slow part, so on some apps we're still working through the backlog. Topic numbers only include the reviews we've finished sorting, never an estimate of the rest. When a review mentions the writer's age, gender, why they use the app or what they paid, we note that too.
- 4
We count, and we show the count
Each topic starts as a plain count: how many sorted reviews bring it up. We only count reviews rated 1, 2 or 3 stars as complaints. To compare an app with others, we turn that count into a percentage: complaints that mention the topic, out of all 1- to 3-star reviews that say what went wrong. Vague ones, like a one-word "Bad", are left out of that total, so an app that gets a lot of them doesn't look like it has fewer problems. The typical app is the median percentage among apps with at least 30 such reviews on the same store. When an app's reviews come from both stores, we blend the two medians by how many of its reviews came from each. Reviewers who rate an app 4 or 5 stars sometimes bring up the same subject to say the opposite — that the profiles looked real — so we show those separately instead of folding them in.
- 5
Readers add their own reviews and votes
You can post a review on any app page, and vote on who uses the app: age, gender, what people use it for and what they spend. We hold each posted review until we've looked at it, then it shows up next to the store reviews and counts toward the review total for that app. Votes are added to the age, gender, purpose and spending figures we take from reviews. Posted reviews go through the same sorting as store reviews, so once we've sorted one, it joins the topic numbers too.
What the numbers won't tell you
It's a count of what reviewers said, not a finding about the company
When a reviewer writes that they ran into a fake profile, we count that they wrote it. We cannot check whether the profile was fake, and we do not say the app made it. Nothing on this site is a finding that any company or person broke the law.
People who had a bad time write more often
That's true of every review site, and it's true here. Someone who quietly enjoyed the app is less likely to post. Read our numbers as "how often reviewers brought this up", not "how often this happens to users".
A low count doesn't mean few problems
It can also mean few reviews. Some apps have only a few dozen. On the very large apps we only have the most recent reviews, so a problem people stopped writing about a few years ago won't show up.
Small numbers move a lot
Below 30 sorted reviews, we show the count and no percentage. Two complaints out of five is 40%, and it means almost nothing.
The store average and our average are different numbers
The stores average every rating, including the millions of people who tapped a star and wrote nothing. We can only average the reviews we count, and those skew lower because people who write are more often annoyed. Both numbers are on the page. Neither one is wrong; they're counting different things.
Why there's no "fake review score" here
We built one. It gave every app a percentage that was supposed to mean "how much of this looks planted". Then we checked it against every app we track and more than half a million reviews, and found that the score tracked one thing almost perfectly: how many five-star reviews the app had. Rank the apps by our score and rank them by their share of five-star reviews, and you get nearly the same order.
That's not a detector. An app with a lot of happy users would score as suspicious, and a Christian dating app with no sign of anything wrong came out in our "caution" band. We tried three other ways of measuring it. Each one produced a different ranking of the same apps, which is what it looks like when none of them is measuring what you set out to measure.
So we took the score down. What's left is the part we can stand behind: the reviews, the count, and the wording people actually used. If we find a signal that survives being checked, we'll publish it and show the check.