Google Cloud Platform Blog: Solutions

New GitHub repo: Using Firebase to add cloud-based features to games built on Unity

2018年6月29日星期五

By Dane Zeke LiergaardGoogle Cloud PlatformFirebaseFirebase Unity SolutionsFirebaseFirebase Console Introducing Firebase Leaderboard

Firebase_LeaderboardLeaderboardController MonoBehaviourprivate void Start() { this.leaderboard = FindObjectOfType(); leaderboard.FirebaseInitialized += OnInitialized; leaderboard.TopScoresUpdated += UpdateScoreDisplay; leaderboard.UserScoreUpdated += UpdateUserScoreDisplay; leaderboard.ScoreAdded += ScoreAdded; MessageText.text = "Connecting to Leaderboard..."; }public void AddScore(string userId, int score) { leaderboard.AddScore(userId, score); }hereDemo Scene We want to hear from youthe github repothis Google Group

Three steps to prepare your users for cloud data migration

2018年5月14日星期一

By Paul Williams, Strategic Cloud Engineer 1. Understand the scope

Where is the data coming from?

How much data do we process?

What tools do we use to consume and analyse the data?

What happens to the output that we collect?

2. Establish champions

Training challenges. Mass training is impersonal and can’t be tailored per team. Champions can deliver custom training that will hit home with their team.

Transition difficulties. Individual struggles by team can be hard to track and manage. By giving each team a voice through their champion, users will feel more involved in the project, and their issues are more likely to be addressed, reducing friction in the final stages.

Overloaded support teams. Champions become the voice of the project within the team too. This can have the effect of reducing support workload in the days, weeks and months during and after a migration, since the champion can be the first port of call when things aren’t running quite as expected.

3. Promote the cloud transformation

Promote and develop the understanding of having the power of the cloud behind the data. It’s an opportunity to ask questions of data that might otherwise have been locked away before, whether behind time constraints, or incompatibility with software, or even a lack of awareness that the data was even available to query. By combining data sets, can you and your teams become more evidential, and get better results that answer deeper, more important questions? Invariably, the answer is yes.

In the case that an existing tool will continue to be used, it will be invaluable to provide teams with new data locations and instructions for reconfiguring applications. It is important that this is communicated, whether or not the change will be apparent to the user. Undoubtedly, some custom configuration somewhere will break, but you can reduce the frustration of an interruption by having the right information available.

By having teams develop and build new tooling early, rather than during or after migration, you’ll give them the ability to play with, learn and develop the new tools that will be required. This can be on a static subset of data pulled from the existing setup, creating a sandbox where users can analyze and manipulate familiar data with new tools. That way, you’ll help drive driving the adoption of new tools early and build some excitement around them. (Your champions are a good resource for this.)

Defining SLOs for services with dependencies - CRE life lessons

2018年5月7日星期一

By Robert van Gent, Customer Reliability Engineer and Cody Smith, Site Reliability Engineerprevious episodeCRE Life Lessonschapter in the SRE bookGoogle Cloud Platform service discovery.

Some of your dependencies may not even have SLOs, or their SLOs may not capture how you're using them.

The effect of a dependency's SLO on your service isn't always straightforward. In addition to the "hard" vs "soft" vs "degraded" impact discussed above, your code may complicate the effect of a dependency's SLOs on your service. For example, you have a 10s timeout on an RPC, but its SLO is based on serving a response within 30s. Or, your code does retries, and its impact on your service depends on the effectiveness of those retries (e.g., if the dependency fails 0.1% of all requests, does your retry have a 0.1% chance of failing or is there something about your request that means it is more than 0.1% likely to fail again?).

How to combine SLOs of multiple dependencies depends on the correlation between them. At the extremes, if all of your dependencies are always unavailable at the same time, then theoretically your unavailability is based on the max(), i.e., the dependency with the longest unavailability. If they are unavailable at distinct times, then theoretically your unavailability is the sum() of the unavailability of each dependency. The reality is likely somewhere in between.

Services usually do better than their SLOs (and usually much better than their service level agreements), so using them to estimate your downtime is often too conservative.

Risky businessSRECon18 Americasslidesrisk analysis spreadsheeta previous CRE Life Lessons blog postspreadsheet1

Trying to remove it from your critical path

Making it more reliable; e.g., running multiple copies and failing over between them

Automating manual failover processes

Replacing it with a more reliable alternative

Sharding it so that the scope of failure is reduced

Adding retries

Increasing (or decreasing, sometimes it is better to fail fast and retry!) RPC timeouts

Adding caching and using stale data instead of live data

Adding graceful degradation using partial responses

Asking for an SLO that better meets your needs

123balance velocity and risk1 If you're interested, The Calculus of Service Availability has more in-depth discussion about modeling risks from dependencies, and strategies for mitigating them.

Scale big while staying small with serverless on GCP — the Guesswork.co story

2018年5月1日星期二

By Mani Doraisamy, founder at Guesswork.co and Google Developer Expert[Editor’s note: Mani Doraisamy built two products—Guesswork.co and CommerceDNA—on top of Google Cloud Platform. In this blog post he shares insights into how his application architecture evolved to support the changing needs of his growing customer base while still staying cost-effective.]ZaloraGaleries Lafayette GroupDaraz

Stay small. As a small team of three developers, we didn’t want to add any additional personnel even if we needed to scale up for a huge volume of users.

Stay profitable. Our revenue is based on the performance of our recommendation engine. Instead of a recurring fee, customers pay us a commission on sales to their users that come from our recommendations. This business model made our application architecture and infrastructure costs a key factor in our ability to turn a profit.

Embrace constraints. In order to increase our development velocity and stay flexible, we decided to trade off control over our development stack and embrace constraints imposed by managed cloud services.

Choosing a database: The Three MusketeersGoogle Cloud Platform

Cloud Datastore for our transactional database, because it can support high number of writes. In our case, the user events are in the billions and are updated in real time into Cloud Datastore.

BigQuery to analyze user behaviour. For example, we understand from BigQuery that users coming from a fashion blog usually buy a specific type of formal shoes.

Vision API to analyze product images and categorize products. Since we work with e-commerce companies across different geographies, the product names and descriptions are in different languages, and categorizing products based on images is more efficient than text analysis. We use this data along with user behaviour data from BigQuery and Cloud Datastore to make product recommendations.

First take: the App Engine approachApp EngineCloud Dataflow

We compare the price we receive in the user event with the product master and determine if there is a difference.

If there is a difference, we propagate it to the analytical and machine learning databases to trigger an alert and reflect that change in the product recommendation.

Take two: the Cloud Functions approach

Use memcache to load the product master in memory and compare the price/stock for every user event. With this option, we had no guarantee that memcache would be able to hold so many products in memory. So, we might miss a price change and end up with inaccurate product prices.

Use Cloud Firestore to record user events and product data. Firestore has an option to trigger Cloud Functions whenever there’s a change in value of an entity. In our case, the price/stock change automatically triggers a cloud function that updates the analytical and machine learning databases.

With Firestore, we replaced both App Engine and Datastore. Firestore was able to accept user requests directly from a browser without the need for a front-end service like App Engine. It also scaled well like Datastore.

We used Cloud Functions not only as a way to trigger price/stock alerts, but as an orchestration tool to synchronize data between Firestore, BigQuery and our recommendation engine.

What we gained

We increased our sales commissions by improving recommendation accuracy

The best thing that happened in this new version was the ability to A/B test new algorithms. For example, we found that users who browse e-commerce sites with an Android phone are more likely to buy products that are on sale. So, we included user’s device as a feature in the recommendation algorithm and tested it with a small sample set. Since, Cloud Functions are loosely coupled (with Cloud Pub/Sub), we could implement a new algorithm and redirect users based on their device and geography. Once the algorithm produced good results, we rolled it out to all users without taking down the system. With this approach, we were able to continuously improve the accuracy of our recommendations, increasing revenue.

We reduced costs by optimizing our algorithm

As counter intuitive it may sound, we also found that paying more money for compute didn't improve accuracy. For example, we analyzed a month of a user’s events vs. the latest session’s events to predict what the user was likely to buy next. We found that the latest session was more accurate even though it had less data points. The simpler and more intuitive the algorithm, the better it performed. Since Cloud Functions are modular by design, we were able to refactor each module and reduce costs without losing accuracy.

We reduced our dependence on external IT teams and signed more customers

We work with large companies and depending on their IT team, it can take a long time to integrate our solution. Cloud Functions allowed us to implement configurable modules for each of our customers. For example, while working with French e-commerce companies, we had to translate the product details we receive in the user events into English. Since Cloud Functions supports Node.js, we enabled scriptable modules in JavaScript for each customer that allowed us to implement translation on our end, instead of waiting for the customer’s IT team. This reduced our go-live time from months to days, and we were able to sign up new customers who otherwise might not have been able to invest the necessary time and effort up-front.

headless ChromeCompute Engine Let a thousand flowers bloom

Building reliable deployments with Spinnaker, Container Engine and Container Builder

2017年10月9日星期一

By Vic Iglesias, Cloud Solutions Architect
primitives to help you deployContainer EngineContainer BuilderSpinnaker

continuous delivery pipeline from scratch using Container Builder and Spinnaker

Spinnaker solutionSpinnaker’s pipeline stages

Customizing Stackdriver Logs for Container Engine with Fluentd

2017年10月9日星期一

By John La Barge, Cloud Solutions Architect
Google Cloud PlatformContainer EngineStackdriverFluentdthis tutorial

Using Stackdriver Logging for dedicated game server instances: new tutorial

2017年10月6日星期五

Joseph Holley, Cloud Solutions Architect
tutorialStackdriver Logging@gcpjoe

Solutions guide: Preparing Container Engine environments for production

2017年6月8日星期四

By Vic Iglesias, Cloud Solutions ArchitectGoogle Cloud PlatformContainer Enginecreated a guide

full solution guidedocumentation

Evaluating Cloud SQL Second Generation for your mobile game

2016年9月26日星期一

Posted by Joseph Holley, Gaming Solutions Architect, Google Cloud PlatformACID-compliantnew solutions documentsource code for the frameworkdocumentationgaming solutions page

Web serving on Google Cloud Platform: an overview

2016年9月6日星期二

Posted by Jim Travis, Senior Lead Technical Writer
Google Cloud PlatformServing Websitesdocumentationsolutionstutorials

Option	Product	Summary
Static website	Google Cloud Storage	Deliver static web pages and assets from a Cloud Storage bucket. This is the simplest option on GCP, and you get automatic scaling with no additional effort.
Virtual machines	Google Compute Engine	Install, configure, and maintain your own web hosting stack. You have control of every component, but you also have all the responsibility to keep things running. You also must decide how to provide for load balancing and scalability, from a variety of options.
Containers	Google Container Engine	Use container technology to package your dependencies with your code for easier deployment. Then, use Container Engine to manage clusters of your containers.
Managed platform	Google App Engine	Focus on your code, deploy to App Engine, and let Google manage the systems for you. You have a choice of the standard environment, which prescribes the languages and runtimes that you can use, and the flexible environment, that gives you additional options but requires some self-management.

free trial

Updated and expanded: Google Cloud Platform for AWS Professionals

2016年7月29日星期五

Posted by Peter-Mark Verwoerd, Solutions Architect
guideGoogle Cloud PlatformGoogle Compute EngineGoogle Container Engine

Amazon ECS vs. Google Container Engine at a glance

How Amazon Elastic MapReduce compares to Google Cloud Dataproc and Cloud Dataflow

saidfree trial

A better way to bootstrap MongoDB on Google Cloud Platform

2016年6月21日星期二

Posted by Sandeep Parikh, Cloud Solutions Architect
Google Cloud Platformstandardcustomizableblazing fast persistent disksGoogle Compute EngineGoogle BigQuery,Cloud DataprocCloud Dataflow

Creating Compute Engine instances and manually installing/configuring MongoDB

Using Google Cloud Launcher to quickly create and test drive a MongoDB replica set

Provisioning Compute Engine instances and using MongoDB Cloud Manager to install, configure and manage MongoDB deployments

documentationCloud Deployment Manager templatesMongoDB Cloud Manager

sharded cluster@crcsmnky

Build your own scalable, location analysis platform with Google Cloud Platform and Maps APIs

2016年6月20日星期一

Posted by Ed Boiling, Solution Architect, Google Maps for Work and Grace Mollison, Solutions Architect, Google Cloud PlatformGoogle Cloud Pub/SubGoogle BigQuery Google Maps APIs

The sample solution retrieves data using BigQuery and renders it as a heat map indicating density.

——solution documenttutorialfork the code on GitHub

Your Google Cloud Platform compute options, explained

2016年6月7日星期二

Posted by Alex Barrett, Editor, Google Cloud Platform BlogGoogle Cloud PlatformGoogle Compute EngineGoogle App EngineGoogle Container EngineChoosing a Computing Option

Learn how Google Cloud Platform services map to Amazon Web Services (AWS)

Check out Google Cloud Platform for AWS professionals: A guide designed to equip users familiar with AWS with the key concepts required to get started with Google Cloud Platform

Learn how Google Cloud Platform services map to Microsoft Azure services

@googlecloud

IAM best practice guides available now

2016年3月29日星期二

Google Cloud Identity & Access ManagemenGoogle Cloud Platform

Using IAM Securely

Designing Resource Hierarchies

Understanding Service Accounts

Using IAM Securely

Least privilege - A set of checks that assist you in restricting your users or applications to not do more than they're supposed to.

Managing Service Accounts and Service Account keys - Provides pointers to help you manage both securely.

Auditing - This covers practices that include reminding you to use Audit logs and cloud logging roles

Policy Management - Some checks to ensure that you're implementing and managing your policies appropriately.

Organization level. The Organization resource represents your company. IAM roles granted at this level are inherited by all resources under the organization.

Project level. Projects represent a trust boundary within your company. Services within the same project have a default level of trust. For example, App Engine instances can access Cloud storage buckets within the same project. IAM roles granted at the project level are inherited by resources within that project.

Resource level. In addition to the existing Google Cloud Storage and Google BigQuery ACL systems, additional resources such as Google Genomics Datasets and Google Cloud Pub/Sub topics support resource-level roles so that you can grant certain users permission to a single resource.

Designing Resource HierarchiesUnderstanding Service Accounts

What resources can the service account access?

What permissions does it need?

Where will the code assuming the identity of the service account be running: on Google Cloud Platform or on-premises?

GCP-iam-feedback@google.comPosted by Grace Mollison, Solutions Architect

TensorFlow machine learning with financial data on Google Cloud Platform

2016年3月2日星期三

TensorFlow Machine Learning with Financial Data on Google Cloud PlatformGoogle Cloud DatalabGoogle BigQueryPosted by Corrie Elston, Solutions Architect, Google Cloud Platform

How to build your own recommendation engine using machine learning on Google Compute Engine

2016年3月1日星期二

recommendation engineGoogle Cloud Platform

Multi-tenant, redundant datacenters

Custom instance types

Global network

Years of experience in software infrastructure innovation, such as MapReduce, BigTable, Dremel, Flume, and Spanner.

Antvoicesolution article

Front-End: By deploying a simple application on Google App Engine, a user can see a page with top recommendations. You can take it from there, easily building a strong and scalable web platform that can manage one to several millions of users, with minimum operations.

Storage: The solution uses Google Cloud SQL, our managed MySQL option. A commonly used database in the ecommerce domain, this database integrates well with MLlib, a machine learning library.

Machine learning: Using Google Cloud Dataproc or bdutil, two options that simplify deployment and management of Hadoop/Spark clusters, you'll deploy and run MLlib-based scripts.

Timeliness concerns, such as real-time, near-real-time, and batch data analysis. This information can help you understand your options for how quickly you can present recommendations to the user and what it takes to implement each option. The sample solution focuses mainly on a near-real-time approach.

Filtering methods, such as content-based, cluster and collaborative filtering. You'll need to decide exactly what information goes into making a recommendation, and these filtering methods are the common ones in use today. The sample solution focuses mainly on collaborative filtering, but a helpful appendix provides more information about the other options.

Download and follow the instructions on the Google Cloud Platform Github page

“Recommend” this solution to your friends.

Posted by Matthieu Mayran, Cloud Solutions Architect

How to build mobile apps on Google Cloud Platform

2016年2月10日星期三

Scale automatically to meet demand

Automatically synchronize data across devices

Handle the offline case gracefully

Send notifications and messages

Build mobile apps using Google Cloud Platform Real-time data synchronization with Firebase

Add managed computation to Firebase apps with Google App Engine

Add flexible computation to Firebase with App Engine Managed VMs

Automatically generate client libraries with App Engine and Google Cloud Endpoints

Have full control with Compute Engine and REST or gRPC

How to build backend services for mobile apps- Posted by Syne Mitchell, Technical Writer, Google Cloud Platform