Google Cloud Platform Blog: December 2016

Stackdriver Trace + Zipkin: distributed tracing and performance analysis for everyone

Tuesday, December 27, 2016

Posted by Morgan McLean, Product Manager for Stackdriver TraceEditor's Note: You can now use Zipkin tracers with Stackdriver Trace. Go here to get started.Google Cloud PlatformDapper distributed tracing system, Stackdriver TraceGoogle App EngineNode.jsJavaGoan APITwitter open-sourced in 2012we’re releasing a Zipkin server

Setup InstructionsUsing Stackdriver with Zipkin Collector guidea list of the most popular Zipkin tracers FAQQ: What does this announcement mean if I’ve been wanting to use Stackdriver Trace but it doesn’t yet support my language?Q: What does this announcement mean if I currently use Zipkin?

Q: What are the limitations of using the Stackdriver Trace Zipkin Collector?

Zipkin tracers must support the correct Zipkin time and duration semantics.

Zipkin tracers and the Stackdriver Trace instrumentation libraries can’t append spans to the same traces, meaning that traces that are captured in one library won’t contain spans for services instrumented in the other type of library. For example:

In this example, requests made to the Node.js web application will be traced with the Zipkin library and sent to Stackdriver Trace. However, these traces do not contain spans generated within the API application or for the RPC calls that it makes to the Database. This is because Zipkin and Stackdriver Trace use different formats for propagating trace context between services. For this reason we recommend that projects wanting to use Stackdriver Trace either exclusively use Zipkin-compatible tracers along with the Zipkin Connector, or use instrumentation libraries that work natively with Stackdriver Trace (like the official Node.js, Java or Go libraries).Q: Will this work as a full Zipkin server?Q: How much does Stackdriver Trace cost?Q: Can I use Stackdriver Trace to analyze my AWS, on-premises, or hybrid applications or is it strictly for services running on Google Cloud Platform? Wrapping upAdrian Colepublished on GitHub

Happy holidays and an anomalously great New Year

Thursday, December 22, 2016

Posted by Alex Barrett, Editor, Google Cloud Platform BlogTwas the night before Christmas and all through the CloudNot a creature was deploying; it wasn't allowed.The servers were all hosted in GCP or AWSAnd Stackdriver was monitoring them so no one was stressed.

The engineers were nestled all snug in their bedsWhile visions of dashboards danced in their heads.When then from my nightstand, there arose such a clatter,I silenced my phone and checked what was the matter.

Elevated error rates and latency through the roof?At this rate our error budget soon would go poof!The Director OOO, the CTO on vacation,Who would I find still manning their workstation?

Dutifully, I opened the incident channel on SlackAnd couldn't believe when someone answered back.SClaus was the user name of this tireless engineer.I wasn't aware that this guy even worked here.

He wrote, "Wait while I check your Stackdriver yule Logs . . .Yep, it seems the errors are all coming from your blogs."Then in Error Reporting, he found the root cause"Quota is updated. All fixed. :-)" typed SClaus.

Who this merry DevOps elf was, I never shall know.For before we did our postmortem, away did he go.Just before vanishing, he took time to write,"Merry monitoring to all and to all a silent night!"

Google Cloud Architect Certification (beta) - registration now open

Thursday, December 22, 2016

Posted by Jennifer Tollar, Certification Program Manager
Google Certified ProfessionalGoogle Certified Professional - Cloud ArchitectGoogle Cloud Platform

Google Cloud Architect Certification Exam Guide

Save 40% on the cost of certification.

Prove early adoption by claiming a low certificate number if you pass.

Get exclusive access to the Certification Lounge at Google Cloud Next ’17 if you pass.

Google Cloud Platform icons and sample architectural diagrams, for your designing pleasure

Wednesday, December 21, 2016

Posted by Miles Ward, Global Head of Solutions, Google Cloud PlatformGoogle Cloud Platformcloud.google.com/icons

How to enable Google Stackdriver Logging, Monitoring and Error Reporting for .NET apps

Wednesday, December 21, 2016

Posted by Jonathan Simon, Developer Relations
Google StackdriverKhan Academy and WixStackdriver LoggingStackdriver MonitoringGoogle Cloud PlatformStackdriver Diagnostics Logging agentGoogle Compute EngineGoogle App EngineLogging agentinstructionstextPayload: "Successfully sent to Google Cloud Logging API"Stackdriver Logs Viewer

Monitoring agentMonitoring agentMetrics Listinstallation instructions—

From the Monitoring dashboard click "Create Check" under "Uptime checks."

Enter the details for the new Uptime check including Name, Check Type, Resource Type, Hostname and Path and specify how often to run the Uptime check under the "Check every" field.

Click "Save."

Logging custom events for .NET Applicationsyour Visual Studio projectLog4Net XML configuration section<configuration> <configSections> <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" /> </configSections> <log4net> <appender name="CloudLogger" type="Google.Cloud.Logging.Log4Net.GoogleStackdriverAppender,Google.Cloud.Logging.Log4Net"> <layout type="log4net.Layout.PatternLayout"> <conversionPattern value="%-4timestamp [%thread] %-5level %logger %ndc - %message" /> </layout> <projectId value="YOUR-PROJECT-ID" /> <logId value="mySampleLog" /> </appender> <root> <level value="ALL" /> <appender-ref ref="CloudLogger" /> </root> </log4net>Global.asax.cslog4net.Config.XmlConfigurator.Configure();protected void Application_Start() { GlobalConfiguration.Configure(WebApiConfig.Register); // Configure log4net to use Stackdriver logging from the XML configuration file. log4net.Config.XmlConfigurator.Configure(); }using log4net;Stackdriver Logs Viewerour application// Retrieve a logger for this context. ILog log = LogManager.GetLogger(typeof(WebApiConfig)); // Log some information to Google Stackdriver Logging. log.Info("Hello World.");

installing and using the Logging client Error Reporting for .NET ApplicationsError Reporting APIusing Google.Cloud.Diagnostics.AspNet;.NET web apppublic static void Register(HttpConfiguration config) { // Add a catch all for the uncaught exceptions. string projectId = "YOUR-PROJECT-ID"; string serviceName = "NAME-OF-YOUR-SERVICE"; string version = "VERSION-OF-YOUR-SERVICE"; // Add a catch all for the uncaught exceptions. config.Services.Add(typeof(IExceptionLogger), ErrorReportingExceptionLogger.Create(projectId, serviceName, version)); }Error Reporting

installing and using the Stackdriver Diagnostics ASP.NET NuGet package Try it outdeploy a .NET application to Google Cloud

Using load shedding to survive a success disaster - CRE life lessons

Monday, December 19, 2016

Posted by Dave Rensin, Director of Customer Reliability Engineering, and Adrian Hilton, Software Engineer, Site Reliability EngineeringEditor’s note: Just because something is a good problem to have, doesn’t mean it’s not a problem. In this latest installment of the CRE life lessons series, we learn about techniques that the Google Site Reliability Engineering team uses to handle too much of a good thing (traffic) with grace — and how you can apply them to your own code running on Google Cloud Platform (GCP).prevent an accidental DDoS— thinkshopingoad sheddingserve nominal capacity, regardless of how much traffic is being sent to it Procrustean load sheddingProcrustesnndef addRequest(self, r): HARD_QUOTA = 45 SOFT_QUOTA = 25 STEPS = 10 divisor = (HARD_QUOTA - SOFT_QUOTA) / STEPS self.received += 1 self.req_modulus = (self.req_modulus + 1) % STEPS # Are we overloaded? load = self.getLoad() # Become progressively more likely to reject requests # once load > soft quota; reject everything once load # hits hard limit. threshold = int((HARD_QUOTA - load) / divisor) if self.req_modulus < threshold: # We're not too loaded self.active_requests.append(r) self.accepted += 1 else: self.rejected += 1

Ranking requests for criticality and cost

The cost to perform the work (the direct cost)

The cost to not perform the work (the opportunity cost)

Denominate your costs in terms of your scarcest resource. If CPU is the scarcest thing in your system then use that to express all of your costs. If it’s revenue or profit then use that. At Google, for example, we sometimes use engineering hours as a measure of cost because we perceive engineering time as more scarce than dollars.

Get everyone to agree on the units before you start ranking request types. Different parts of your business will have different views of the costs of dropping traffic. The ads team might value the dollars in lost revenue for not serving a piece of content while your marketing team might value the total number of users that can simultaneously access your application. The UX team, on the other hand, might think that latency is the most important thing since laggy UIs make users grumpy. The point is that this all gets settled by deciding on the denominating units first!

establishing your criticalityminimize the aggregate opportunity costweightedscaledbucketsclasses Setting criticality

An explicit field in the request specifying the bucket.

Bucketing by the hostname, which lets you "black-hole" low-priority traffic in overload situations by using DNS to point to a sacrificial server. This is a big hammer, but occasionally life-saving because it can stop requests from reaching your overloaded service in the first place.

The URL path, which is fairly cheap to check though does require some extra processing by your front-end service.

User ID, and whether it belongs to a specific group, e.g.,"paying customers" (highest), "logged in users" (medium-high), "logged-out users" (medium-low), "known robot accounts" (lowest). This allows the most precise bucketing, but is more expensive to check for each request.

Criticality changes over time Soft quotas vs. hard quotas—clients who have exceeded their quotas should not be throttled if the system has remaining capacity— Optimistic and pessimistic throttlingwhichoptimisticpessimisticany— Throttling as a signal Case study

Wrapping up—May your queries flow and your pagers stay silent .

Automated node management, stateful apps and HIPAA compliance come to Google Container Engine

Monday, December 19, 2016

Posted by David Aronchick, Product Manager, Google Container EngineKubernetes 1.5 releaseGoogle Cloud PlatformGoogle Container Engine

Auto-upgrade and auto-repair for nodes simplify on-going management of your clusters

Simplified cross-cloud federation with support for the new "kubefed" tool

Automated scaling for key cluster add-ons, ensuring improved uptime for critical cluster services

StatefulSets (originally called PetSets) in beta, enabling you to run stateful workloads on Container Engine

HIPAA compliance allowing you to run HIPAA regulated workloads in containers (after agreement to Google Cloud’s standard Business Associate Agreement).

GroupBy uses Container Engine to support continuous delivery of new commerce application capabilities for their customers, including retailers such as The Container Store, Urban Outfitters and CVS Health.

“Google Container Engine provides us with the openness, stability and scalability we need to manage and orchestrate our Docker containers. This year, our customers flourished during Black Friday and Cyber Monday with zero outages, downtime or interruptions in service thanks, in part, to Google Container Engine.” - Will Warren, Chief Technology Officer at GroupBy.

MightyTV ported their workloads to Container Engine to power their video recommendation engine, reducing their cost by 33% compared to running on traditional virtual machines. Additionally, they were able to remove a third-party monitoring and logging service and let go of maintaining Kubernetes on their own.

— —kubernetes-users-mailing listkubernetes-users Slackfree trial here

Google joins the Cloud Foundry Foundation

Thursday, December 15, 2016

Posted by Brian Stevens, Vice President, Google CloudGoogle Cloud Platformbuild the most open cloudCloud Foundry Foundation

Building on successBOSH Google CPI releaseCloud Foundry on GCPOpen Service Broker APIintegration with tools like Google Stackdrivercustom service brokers

Google Cloud Vision API

Google Cloud Speech API

Google Cloud Natural Language API

Google Translation API

What’s next

Building scalable private services with Internal Load Balancing

Thursday, December 15, 2016

Posted by Prajakta Joshi, Product Manager, Cloud NetworkingInternal Load BalancingGoogle Cloud Load BalancingGlobal Load BalancingNetwork Load BalancingNiantic deployed HTTP(S) LBInternal Load Balancing architectureAndromeda

(click to enlarge)

Internal Load Balancing features

(click to enlarge)

Configure a private RFC1918 load-balancing IP from within your virtual network;

Load balance across instances in multiple availability zones within a region;

Configure session affinity to ensure that traffic from a client is load balanced to the same backend instance;

Configure high-fidelity TCP, SSL(TLS), HTTP or HTTPS health checks;

Get instant scaling for your backend instances with no pre-warming; and

Get all the benefits of a fully managed load balancing service. You no longer have to worry about load balancer availability or the load balancer being a choke point.

Configuring Internal Load BalancinghereThe (use) case for Internal Load Balancing1. Scaling your internal services

(click to enlarge)

2. Building multi-tier applications on GCP

(click to enlarge)

3. Delivering high availability and scale for virtual appliances

(click to enlarge)

What’s next for Internal Load Balancingregional instance groupstutorialdocumentationfeedback

Google partners with Improbable to support next generation of video games

Tuesday, December 13, 2016

Nan Boden, Head of Global Technology Partners, Google CloudAlphaGoPokemon GOImprobableSpatialOSGoogle Cloud DatastoreGoogle Compute EngineBossa StudiosWorlds Adrift

A collision of two fully customized ships flying through the procedurally generated and persistent universe of Worlds Adrift. Read about the game’s origin story and technical details of its physics.

SpatialOS.com

Announcing new Google Cloud Client Libraries for four key services

Monday, December 12, 2016

Posted by Omar Ayoub, Product ManagerBigQueryGoogle Cloud DatastoreStackdriver LoggingGoogle Cloud StorageC#GoJavaNode.jsPHPPythonRuby Finding client libraries fastcloud.google.comBigQuery

Client Libraries

Client libraries you’ll want to useNode.js client library reference

stream data into BigQuery

Next stepsfile issues on GitHubask questions on StackOverflow