Google Cloud Platform Blog
Product updates, customer stories, and tips and tricks on Google Cloud Platform
Scaling SongPop to 60 million users with App Engine and Google Cloud Storage
Monday, February 4, 2013
Continuing our
Developer Insights series
, today’s guest bloggers are Olivier Michon, CTO, and
Alexis Hanicotte, software engineer, from
Fresh Planet
,
maker of the popular mobile application
SongPop
. SongPop is a social application where players compete to be the fastest to guess the name of a song or artist.
SongPop is a social mobile app where players compete on who will be the fastest to guess a songs’ artist or title. It is a huge success for our very small team of just 6 engineers. We now have more than 60 million users, were the number 5 most downloaded iOS game of 2012, and went from 0 to more than 10k queries/second on our servers in less than 6 months. This has been made possible in large part because we run on
Google App Engine
.
App Engine allows us to quickly prototype, iterate and release our games. We’ve been accumulating experience with the platform since 2009, but really saw the power of autoscaling once SongPop became a hit.
Our experience scaling with App Engine
SongPop was released in May 2012. During our growth to our first 100,000 daily active users (DAU), our App Engine backend scaled smoothly. This allowed us to spend our time making actual improvements to the game experience, while our user growth continued at a rapid pace.
We opened a
Premier Account
with App Engine around the time when we reached 100,000 DAU which gave us access to live customer support. It came in handy when we encountered two downtimes, with one lasting just 10 minutes and the other for 1 hour. But we were able to promptly reach Premier Support and had Google engineers investigate these issues with us.
Once we reached 500,000 DAU, we applied a variety of optimization ideas to reduce latency. For example, we used to have user data spread over many models, but we combined them into a single entry in the
Datastore
to reduce read operations. We also often needed the list of a user’s opponents, so instead of querying every time, we cached this result using
Memcache
. It only took one engineer and just 4 days of work to reduce the latency with these optimizations.
As we reached the milestone of 1 million DAU, some Datastore queries (used to find random opponents in the game) showed high latencies and a high rate of timeouts. We had to enforce deadlines, implement better fallbacks and identified, with the help of Premier Support, that degraded performance came from the fact that our queries were relying on many different indexed properties. So the solution was simple - either add a composite index with all the properties we needed or combine into a single one.
Using App Engine + Google Cloud Storage
For each game session, our users need to download song samples in order to play. It is critical that this data gets delivered fast and reliably wherever the user is located. We chose to use
Google Cloud Storage
for this use case. It has proved to provide high performance content delivery, as we have been able to serve 17 terabytes/day of songs and images worldwide.
In addition to its reliability, Cloud Storage is great because of its integration with App Engine. We can easily read and write files from our application to Cloud Storage using the same syntax as we would use to write local files (using Python). We found it intuitive and convenient because you do not have to manage opaque keys to retrieve your files (just use the path you specified), and you can browse your files through a directory-like structure. Cloud Storage also allows you to manage access rights, can be used with Google BigQuery, and it is priced affordably compared to other solutions we considered.
Architecture diagram of SongPop
Long live App Engine!
When we speak to other game developers, we always recommend that they use App Engine. We’ve used other services such as EC2 from Amazon Web Services for other games before, but we’ve found App Engine to be a better service for our needs. We don’t want to spend time setting up servers and load balancing, when we could instead use that time to build great games and let our service provider handle the infrastructure for us.
When we compare the development of SongPop to stories of other apps, we’re thankful that App Engine allowed us to have only one engineer working full-time on the backend portion of our app. Even better, he was able to do additional work on adding new features to the game instead of solely focusing on infrastructure issues. With App Engine, scaling our game was easy.
Others things we want to share
Do not worry if
documented resources limits
and rates look too small, they are just there to make sure one app does not abuse any resource, but they can scale. Most of our limits have been increased by 18-fold! We had days where we made 230,000,000 UrlFetch API calls, for instance.
The
Location headers
is a really great feature, because it is easily accessible for a wide variety of use cases, such as selecting users’ opponents or building their game profile.
Contributed by Olivier Michon and Alexis Hanicotte, FreshPlanet
Posted by Zafir Khan, Product Marketing Manager, Google App Engine
Free Trial
GCP Blogs
Big Data & Machine Learning
Kubernetes
GCP Japan Blog
Firebase Blog
Apigee Blog
Popular Posts
Understanding Cloud Pricing
World's largest event dataset now publicly available in BigQuery
A look inside Google’s Data Center Networks
New in Google Cloud Storage: auto-delete, regional buckets and faster uploads
Enter the Andromeda zone - Google Cloud Platform’s latest networking stack
Labels
Announcements
193
Big Data & Machine Learning
134
Compute
271
Containers & Kubernetes
92
CRE
27
Customers
107
Developer Tools & Insights
151
Events
38
Infrastructure
44
Management Tools
87
Networking
43
Open
1
Open Source
135
Partners
102
Pricing
28
Security & Identity
85
Solutions
24
Stackdriver
24
Storage & Databases
164
Weekly Roundups
20
Feed
Subscribe by email
Demonstrate your proficiency to design, build and manage solutions on Google Cloud Platform.
Learn More
Technical questions? Check us out on
Stack Overflow
.
Subscribe to
our monthly newsletter
.
Google
on
Follow @googlecloud
Follow
Follow
Follow