Google Cloud Platform Blog: 2013

2013 Year in review: topping 100,000 requests-per-second

Friday, December 27, 2013

2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing his or her highlight from the past year.every single person on the plane

2013 Year in review: making Google Compute Engine Generally Available

Thursday, December 26, 2013

2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing his or her highlight from the past year.Compute Engine Generally AvailableBrightcoveCooladataEviteFishlabsMendelics)scalabilityDataStaxDataTorrentRightscaleSaltStackScalropen-source projects

2013 Year in review: bringing App Engine to the PHP community

Tuesday, December 24, 2013

2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing his or her highlight from the past year.addition of the PHP runtimeGoogle Cloud SQLGoogle Cloud StorageMotherboardlargest livestreamed music event in historySnapchat or Khan Academy

Now Get Programmatic Access to your Billing Data With the New Billing API

Monday, December 23, 2013

Google Cloud ConsoleGoogle Cloud StorageCloud Storage API, CLI tool or Cloud Console file browserACLs on your Cloud Storage bucket

[ {

"lineItemId" : "com.google.cloud/services/compute-engine/StoragePdCapacity",

"startTime" : "2013-11-02T00:00:00-07:00",

"endTime" : "2013-11-03T00:00:00-07:00",

"projectNumber" : "176782591794",

"measurements" : [ {

"measurementId" : "com.google.cloud/services/compute-engine/StoragePdCapacity",

"sum" : "66325032468480000",

"unit" : "byte-seconds"

} ],

"cost" : {

"amount" : "2.383101",

"currency" : "USD"

}

}, {

"lineItemId" : "com.google.cloud/services/compute-engine/VmimageN1Highcpu_8",

"startTime" : "2013-11-02T00:00:00-07:00",

"endTime" : "2013-11-03T00:00:00-07:00",

"projectNumber" : "176782591794",

"measurements" : [ {

"measurementId" : "com.google.cloud/services/compute-engine/VmimageN1Highcpu_8",

"sum" : "44220",

"unit" : "seconds"

} ],

"cost" : {

"amount" : "6.4119",

"currency" : "USD"

}

} ]

2013 year in review: making scalability easy with dedicated memcache

Sunday, December 22, 2013

2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing his or her highlight from the past year.like Snapchat growdedicated memcache

2013 Year in review: taking Google Cloud Platform on the road

Saturday, December 21, 2013

2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing his or her highlight from the past year.

2013 Year in review: pushing the limits of Big Data

Friday, December 20, 2013

2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing his or her highlight from the past year.Big Data Spain conferenceJOIN EACHGROUP EACH BallowLargeResultscorrelation functiondiscovering relationships previously invisible

2013 Year in review: enabling native connections for Cloud SQL instances

Thursday, December 19, 2013

2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing his or her highlight from the past year.native connections for Cloud SQL instancesEurovision Song Contest app running on Google Cloud Platform

2013 Year in review: bringing Offline Disk Import to users around the world

Wednesday, December 18, 2013

2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing his or her highlight from the past year.several international locationsOmNomNom StreetView infrastructure

Best practices for App Engine: memcache and eventual vs. strong consistency

Tuesday, December 17, 2013

Best Practices for App Engine Memcachenew articleConcurrency, performance and migration in memcacheBest Practices for App Engine MemcacheBalancing strong and eventual consistency

Eventual conistency

Strong consistency

articlecloud.google.com/resources

2013 Year in review: giving time back to developers

Tuesday, December 17, 2013

2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing his or her highlight from the past year.Google Cloud Datastore

2013 Year in review: bringing together mobile and cloud

Monday, December 16, 2013

2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing his or her highlight from the past year.Mobile Backend Starterlater iOSAndroid StudioGoogle Cloud Endpoints

Go on App Engine: tools, tests, and concurrency

Friday, December 13, 2013

The goapp toolcommandsgoapp servegoapp deploy$ goapp serve$ goapp deploy$ goapp serve github.com/user/myappmodule$ goapp deploy mymodule.yamlLocal unit testingnative testing packagego testaetest packageLocal Unit Testing for Go documentationBetter concurrency supportmax_concurrent_requestsAutomatic Scaling modulesapplication: maxigopher version: 1 runtime: go api_version: go1 automatic_scaling: max_concurrent_requests: 100google-appengine-go group

Qubole helps you run Hadoop on Google Compute Engine

Thursday, December 12, 2013

This guest post comes form Praveen Seluka, Software Engineer at Qubole, a leading provider of Hadoop-as-a-service. QuboleQubole Data ServiceAuto-scaling and self-managed HadoopPerformance and reliabilityFully integrated tools for Big DataTPC-H datasetbitbucket repositorygit clone 'https://bitbucket.org/qubole/tpch.git'

Apply for our beta program

Outfit 7’s Talking Friends built on Google App Engine, recently hit one billion downloads

Wednesday, December 11, 2013

Today’s guest blogger is Igor Lautar, senior director of technology at Outfit7 (Ekipa2 subsidiary), one of the fastest-growing media entertainment companies on the planet. Its flagship franchise Talking Tom and Friends has achieved over 1.2 billion downloads since its launch in 2010 and continues to grow with 170 million active users each month. In today’s post, Igor explains how the company has been successful building the backends of its entertainment apps on Google App Engine.

Google Cloud PlatformWhy Google App Engine?Google App EngineFrom App Engine to Cloud Datastore and BigQueryGoogle Cloud DatastoreGoogle Cloud MessagingGoogle BigQuery

You can now deliver any-screen streaming media using Google Compute Engine and Wowza Media Systems

Wednesday, December 11, 2013

Today’s guest post comes from Charlie Good, Chief Technology Officer and Co-founder of Wowza Media Systems.

Compute Engine customers purchase a daily, monthly, or perpetual license for Wowza Media Server directly from www.wowza.com;

They then choose a prebuilt Wowza image within Compute Engine to run on their Compute Engine instance. Using this pre-built image, the initial Wowza installation and configuration steps are already completed, so customers can get started more quickly.

www.wowza.com/google-compute-engine

Using Google Compute Engine with open source software

Tuesday, December 10, 2013

recent announcementClient APIs

Ruby: The fog.io cloud API has had support for Compute Engine since version 1.11.0 back in May. Take a look at the Compute Engine docs to get started with Compute Engine and fog. It primarily supports instance operations such as create, destroy and bootstrap.

Python: The Apache libcloud API project has been receiving solid support and updates for Compute Engine since July. It supports a broad set of Compute Engine features including instances, disks, networks/firewalls, and load-balancer support. The handy getting-started demo gives a good code example of how to use libcloud and Compute Engine.

Java: The jclouds cloud API does have Compute Engine support in “labs”. See the jclouds-labs-google repository for work being done to provide Compute Engine support and to elevate the lab into jclouds-core.

PuppetLab’s Puppet: Puppet has been around since 2005 and has evolved from supporting on-premise and hosted-datacenter support to also managing public cloud infrastructure. With the release of Puppet Enterprise 3.1 last month, Puppet’s Cloud Provisioner tool now supports Compute Engine. If you’re more comfortable with Puppet’s domain-specific-language manifest files, you can also use the gce_compute module available at the forge. Savvy puppet users will also be pleased to see that the next version of facter will have extensive support for Compute Engine. Puppet’s roots are open-source and it continues to have a thriving open-source community.

Opscode’s Chef: Chef is another system that’s been around for many years with a strong open-source background and active community. Chef is an automation platform with a modular design that has been extended to support Compute Engine through its knife-google plugin. The plugin gives you the power to create new Compute Engine instances, bootstrap them into your Chef environment, and easily manage those instances and their installed software. Chef’s ohai node attribute discovery tool has also been updated to support Compute Engine instances. Opscode provides a hosted solution that can make managing your Chef environment easier and more care-free.

SaltStack: Continuing the vein of configuration management, one of the newer systems gaining in popularity is Salt. One of Salt’s main design goals was to provide a highly scalable and fast data collection and execution framework for system administration. Recently, its cloud provisioner system was extended to support Compute Engine instances and includes documentation for getting started.

AnsibleWorks’ Ansible: Ansible is the newest configuration management solution on this list and it also embraces a unique design approach. Ansible does not utilize a centralized configuration server nor does it require any agents running on the managed instances. Ansible instead relies on SSH to remotely execute scripts on the managed nodes. As of 1.4, announced recently, Ansible has wide support for Compute Engine features through an inventory plugin and set of new modules.

CoreOS: CoreOS can best be described as a very thin Linux system that provides just enough “OS” to enable the use of Linux containers. CoreOS is combined with etcd, docker, and systemd to allow you to build cluster-like infrastructure on top of standard physical and virtual machines. Thank you to the fine folks at CoreOS for building an image to support Compute Engine.

Docker: Compute Engine v1 has opened the door for additional operating systems, as well as applications like Docker, which require kernel customizations. Docker is an application for running Linux containers and can now be run on your Compute Engine instances. To get started quickly with docker on Compute Engine, take a look at the installation instructions to get up and running with a few simple commands.

Packer: Packer is one of Mitchell Hashimoto’s projects and its goal is to create machine images across multiple platforms from a single configuration. Compute Engine support is under active development and once complete will allow you to easily create a custom Compute Engine image that you can use as the basis for spinning up your Compute Engine instances. Kelsey Hightower, a primary Packer contributor, is leading the effort for Compute Engine support and if you’d like to help, you can find his work over on github.

Vagrant: Vagrant, another of Mitchell’s projects, is primarily a development tool for easily describing and replicating work environments. It differs a bit from a pure configuration management solution because the primary use-case is to quickly spin up a work environment, make changes, and tear it down again when you’re done. Compute Engine support is enabled by a new custom “provider” and instances can be configured with Vagrant’s “provisioner” plugins. Many thanks to Mitchell for hosting a new vagrant-google repository to house the Compute Engine support for Vagrant.

DataTorrent offers massive-scale, real-time stream analytics on Google Compute Engine

Tuesday, December 10, 2013

Today's guest post comes from Amol Kekre, CTO and co-founder of DataTorrent.Get DataTorrent on Google Compute Engine today

Use it free: download and simply install on your Google Cloud Hadoop cluster.

Visit us to learn more and read up on our technology

DataStax Enterprise feels right at home in Google Compute Engine

Monday, December 9, 2013

Today’s guest post comes from Martin Van Ryswyk, Vice President of Engineering at DataStax. DataStaxApache CassandraTMannouncedDataStax EnterpriseTests and results of DataStax Enterprise on Google Compute Enginehere

Operational stability of 100 nodes spread across two physical zones.

Objective: longevity test at 6,000 record per second (60 record/sec/node) for 72 hours.
Results: we saw trouble-free operation, where data tests completed without issue. Replication completed, where data streamed effortlessly across dual zones.

Robustness during a reboot/failure through reconnecting Persistent Disks to an instance.

Objective: measure impact of terminating a node and re-connecting its disk to a new node.
Results: new nodes joined the Cassandra ring without having to be repaired and with no data loss (no streaming required). We did need to manage IP address changes for the new node.

Push the limits of disk performance for a three node cluster.

Objective: measure response under load when approaching the disk throughput limit.
Results: Our tests showed a good distribution of latency during the tests, and 90% of the I/O write times were less than 8ms (see figures below depicting the medium latency and latency distribution). These results were while our load did not exceed the published throughput (I/O) thresholds (see caps for thresholds).

What’s nextFigures for reference

Why We Deployed Zencoder on Google Cloud Platform

Monday, December 9, 2013

Today's guest post comes from Jon Dahl, VP of Encoding Services at Brightcove.Zencoder cloud encoding serviceCloud Platform regionMulti-cloudFast launch timesConsistent performanceFast I/OIntelligent cachingSuper networkThe GA release of Google Compute Engine is big news

Scalr and Google Compute Engine

Sunday, December 8, 2013

Today's guest post is from Sebastian Stadil, CEO of Scalr. The company providies a web-based control panel for cloud infrastructure that serves as an interface between end users and the multiple cloud platforms that they use. In this post, Sebastian discusses benchmarks they conducted to analyze Compute Engine performance.grandcentrixScalr Eurovision song contestNo random hiccups with Google. Only high and stable performance.By the numbers: the benchmarksdispersion

Bandwidth Dispersion

Latency Dispersion

IOPS Dispersion

Are you trying out Google Compute Engine?watch this talk from the OpenStack Summitlearn more about Scalr on our website, or request a POC

Cloud9 IDE on Google Compute Engine

Saturday, December 7, 2013

Today's guest post comes from Ivar Pruijin, Product Manager at Cloud9 - a popular cloud-based IDE. In this post, Ivar discusses their work with Google Compute Engine.Cloud9 IDEGoogle Compute EngineSpeed

Fast & easy Provisioning & Infrastructure Management

What’s next?

Fishlabs architects upcoming game with Compute Engine and RightScale

Saturday, December 7, 2013

Today’s guest blog comes from Alex Pop, a cloud integration engineer at RightScale. RightScale is a Google Cloud Platform partner that enables leading enterprises to accelerate delivery of cloud-based applications while optimizing cloud usage to reduce risk and costs. With RightScale, IT organizations can deliver instant access to a portfolio of public, private, and hybrid cloud services across business units and development teams while maintaining enterprise control. Since 2007, leading enterprises including Pearson International, Intercontinental Hotels Group, and PBS have launched millions of servers through RightScale.RightScale