Blog

Welcome to the surprising second part of our Android release reports. If you missed part one, you can catch up on it here.

In this report, I had expected to write about sales numbers and piracy rates but given the series of mind-boggling events this past week, this will have to wait a little.

After the phenomenal reaction on our first day, things continued quite well, and by day six, we had an unbelievable rating of 5.0 from over two thousand players.

The exact average rating was 4.96 (99.2% 5-stars) but Google Play shows everything above 4.95 as a clean 5.0

We were so excited until, an hour later, we mysteriously lost 438 of our 5-star reviews. Within a moment, over 25% of all our reviews were just gone.

A mystery

Confused, we took to twitter and asked our players whether anyone could confirm that their review had disappeared and, sure enough, minutes later, multiple confirmations came trickling in.

It looked like Google was removing genuine reviews from our Store page, en masse.

A bit of research brought up a story from a year ago (source) that puts the blame on Google’s algorithm for fighting fake reviews and concluding that genuine reviews are also deleted by it as collateral damage. This particular story talked about losing 4-6 reviews a day. We had just lost over 400.

But our troubles weren’t over. The next day, we lost another 430 reviews. Two days later another 655 reviews were silently removed and then another 500+, gone forever.

By our calculations, if Google’s algorithm hadn’t intervened, we would have nearly 3,000 reviews by now. Instead, 13 days after our release, we have 25 fewer reviews (622) than what we had on the first day. Google seems to have quietly removed at least 77% of all our reviews.

We are lucky that despite the culling of thousands of 5-star reviews, only a handful of people rated the game 3-stars or lower and so our average is still a respectable 4.92 but even so, the question of Why? burns in our minds.

Playing fair

We are not engaging in any dubious practices regarding reviews. We don’t incentivize reviews, we don’t buy reviews, we don’t review the game ourselves and we don’t even tell our friends to review it. The only way we encourage reviews is through an in-game reminder on the mobile version which seems to be the industry standard. In short, there is no reason we can think of to explain why our reviews should have disappeared.

Google responds

We reached out to Google about this mystery and they got back to us within a day (thanks!). They told us that our ratings were removed on purpose and pointed us to their Comments and Ratings policy. They also said that they triple-checked that this was done in according to their policies.

Further questions for clarification were politely declined, again pointing to above policy but refusing to be transparent about particular reasons.

The policies page mentioned by Google is divided into two sections.

One section lists specific policies to prevent abuse which contain things like: “Don’t post content that is abusive or hateful or threatens or harasses others” and “Don’t post fake reviews intended to boost or lower ratings”

The other section is labelled guidelines and tips and states things like: “Keep it readable; don’t use excessive capitalization and punctuation” and “Try to include both positives and drawbacks”.

We asked our players if they would be willing to share the review text that was removed from Google. Lots of them just gave the game a 5-star rating without a custom text. It’s puzzling how a review without text can be against a content policy but even more puzzling is that hand-written reviews like the following one, were also deleted:

Awesome little game that is great to play for 5 minutes or 5 hours. Also one of the best (and best working) ports I’ve seen onto mobile and an all round good time.

one of the reviews silently deleted by Google

Can you spot a policy violation?

Unless Google actually treats the guidelines and tips section as policy and sees a review without downsides as a violation there is just no reasonable cause for this review to have been removed. Unfortunately, because Google are not transparent about their practices and have no accountability for their actions, we have no way to oppose this.

Why is Google doing this?

My best guess is that Google uses the same algorithm to identify fake reviews on both free-to-play games and premium games.

I’m sure there are heaps of services out there that would add hundreds of very positive reviews for cheap on a free-to-play title because cheaters could setup hundreds of fake Google Play users and don’t have to fork out money to purchase a game to review it. Google probably invented and tuned the algorithm to remove these kind of fake reviews from free-to-play titles, but for a premium title, where everyone pays 5$ to even be allowed to write a review, this just doesn’t make any sense.

Disrespecting players but respecting pirates

I’m not so upset that our ratings are now 4.92 instead of 4.96, particularly because I assume that this affects everyone on the Google Play store.

What upsets me most is that this completely disrespects our players. Google lets an algorithm and a machine wipe genuine human expressions from history. Around 3,000 people on this planet not only bought and played the game but then spent extra time to tell others about it. A blink later their voice has been erased forever.

It’s also interesting that Google seems content with removing thousands of reviews left by genuine paying customers on their own store front, yet doesn’t have any safe-guards to prevent hundreds of pirate sites showing up on Google when someone searches for ‘download Game Dev Tycoon APK for free’. Surely, an algorithm could help there…

Anyway, more on the topic of the incredible eco-system of pirated APKs, in a future report.

– Patrick Klug
💚 Greenheart Games

If you don’t want to miss future reports, please sign up to our newsletter: