Google Cloud Platform Blog: October 2016

What’s new with Google Cloud Resource Manager, and other IAM news

Friday, October 28, 2016

Posted by Grace Mollison, Solutions Architect
Google Cloud PlatformGoogle Cloud Identity and Access Management FAQplease let us know Google Cloud Resource Manager’s new Organization resourceGoogle Cloud Resource ManagerOrganization resource Project provisioning fun with the Cloud Resource Manager APIproject.create()REST interfaceRPC interfaceclient librariesgcloud library Automating project creation with Python

Create a service account in a designated project under your Organization resource. We recommended a designated project to contain resources that will be used across the projects in your Organization resource. And because service accounts are associated with a project, creating them in a central designated project will help you manage them.

At a minimum the service account needs to have the resourcemanager.projectCreator IAM role. If you need to enable APIs beyond the default, this will require granting the service account the billing user role at the Organization resource level, so that it can attach projects to the organization resource’s billing account. The service account can then enable the required APIs against the project. The billing account must be associated to the organization resource.

def create_client(http=None): credentials = oauth2client.GoogleCredentials.get_application_default() if credentials.create_scoped_required(): credentials = credentials.create_scoped(CRM_SCOPES) if not http: http = httplib2.Http() credentials.authorize(http) return discovery.build(CRM_SERVICE_NAME, CRM_VERSION, http=http)Organization_id = str(YOUR-ORG_NUMERIC_ID) proj_prefix = "your-proj-prefix-" # must be lower case! proj_id = proj_prefix+"-"+str(random_with_N_digits(6)) #************************def List_projects(org_id): crm = create_client() project_filter = 'parent.type:organization parent.id:%s' % org_id print(project_filter) projects = crm.projects().list(filter=project_filter).execute() #while projects is not None: print(projects)def create_project(proj_id): crm = create_client() print "org id in function is :\n" print(organization_id) new_project = crm.projects().create( body={ 'project_id': proj_id, 'name': proj_id, 'parent': { 'type': 'organization', 'id': organization_id } }).execute()assign IAM policiesGoogle Deployment Manager

Decoding the micro-moments of baseball: can you hear the game through data?

Tuesday, October 25, 2016

Posted by Eric Schmidt, Developer AdvocateGoogle Cloud Platform— Google Cloud DataflowGoogle BigQueryGoogle Cloud Datalab—public data set in BigQuerySportradar The Harry Doyle MethodMajor LeagueMr. Baseball, aka Bob Uecker, aka Harry Doyle—

—

@googlecloudMediumhereRetrosheetMLB’s Baseball Savant

Treat Google Cloud Storage like a file system with our new PowerShell provider

Tuesday, October 25, 2016

Posted by Jim Przybylinski and Chris Smith, Software Engineers
Google Cloud Storageup to 99.95% availabilityfees as low as $0.007GB/monthgsutil really, really Navigating Cloud Storage with Cloud Tools for PowerShellCloud Tools for PowerShellCloud SDK for Windowssystem registrySQL Server instancecd, dir, copy, del,catGoogleCloudcdgs:\dirgcloud init PS C:\> Import-Module GoogleCloud WARNING: The names of some imported commands from the module 'GoogleCloud' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. PS C:\> cd gs:\
PS gs:\> dir | Select Name
Name ---- blog-posts chrsmith-demos.appspot.com chrsmith-pictures database-snapshots-prod staging.chrsmith-demos.appspot.com ...cddirSet-Location and Get-ChildItem Populating Google Cloud StoragemkdirSet-Contentgs:\gootoso-test-bucket\folder PS gs:\> mkdir gootoso-test-bucket | Out-Null PS gs:\> Set-Content gs:\gootoso-test-bucket\folder\file.txt ` -Value "Hello, GCS!" PS gs:\> Test-Path gs:\gootoso-test-bucket\folder\file.txt True PS gs:\> cd .\gootoso-test-bucket\folder PS gs:\gootoso-test-bucket\folder> cat file.txt Hello, GCS!Get-GcsBucketNew-GcsObjectCopy-GcsObjectcd Mixing Cmdlets and the PowerShell Provider PS gs:\gootoso-test-bucket\folder> $objs = dir PS gs:\gootoso-test-bucket\folder> $objs[0].GetType().FullName Google.Apis.Storage.v1.Data.Object PS gs:\gootoso-test-bucket\folder> $objs | Read-GcsObject Hello, GCS!
PS gs:\gootoso-test-bucket\folder> Write-GcsObject -Object $objs[0] -Contents "update"PS gs:\> Remove-GcsBucket -Name gootoso-test-bucketCloud Storage API PS gs:\> cd gs:\blog-posts\images PS gs:\blog-posts\images> $objects = dir -Recurse PS gs:\blog-posts\images> $objects | Sort-Object Size -Descending | Select-Object -First 1 -Property Name,TimeCreated,SizePowerShell documentationGitHub

Now shipping: Windows Server 2016 images on Google Compute Engine

Monday, October 24, 2016

Posted by Amruta Gulanikar, Product Manager
Google Cloud PlatformWindows Server 2016 Datacenter EditionGoogle Compute EngineGoogle Compute Engine VM images with Microsoft Windows Server 2016 preinstalledMicrosoft SQL Server

Windows Server 2016 Datacenter Edition

SQL Server Standard 2016 with Windows Server 2016

SQL Server Web 2016 with Windows Server 2016

SQL Server Express 2016 with Windows Server 2016

SQL Server Standard (2012, 2014, 2016) with Windows Server 2012 R2

SQL Server Web (2012, 2014, 2016) with Windows Server 2012 R2

SQL Server Express (2012, 2014, 2016) with Windows Server 2012 R2

and coming soon, SQL Server Enterprise (2012, 2014, 2016) with Windows Server (2012, 2016)

price-to-performance advantagescustomizable VM sizesnetworkingsecuritypricing Getting startedreceive a $300 creditCloud Launcherdetailed documentation

(click to enlarge)

.NET developer solutions.NET client libraryNuGetLearncloud.google.com/windowscloud.google.com/sql-servercontacthear your feedback

Production debugging the easy way, with Stackdriver Debugger GA

Friday, October 21, 2016

Posted by Sharat Shroff, Product Manager
Stackdriver Debuggerbeta launchDebug pageGithub and Bitbucketraw logs associated

Google App EngineJavaPythonNodeGoogle Compute Engine and Google Container Engine JavaPythonNode (experimental)GoStackdriver Debugger APIgcloud debugrequests and feedbackconnect with the Debugger team

Announcing new storage classes for Google Cloud Storage: simplifying the storage and management of hot and cold data

Thursday, October 20, 2016

Posted by Kirill Tropin, Product Manager
Google Cloud Storage—Google Cloud Platform New storage classes for Google Cloud Storage

Cloud Storage Coldline: a low-latency storage class for long-term archiving ColdlineNearlineNearline latency and throughputGoogle Cloud Storage Transfer Servicecontact our sales teamGoogle Cloud Storage Multi-Regional and RegionalGoogle Compute EngineGoogle Cloud Storage Multi-Regional—

We use Google Cloud Platform, including Google Cloud Storage and Compute Engine along with Fastly, for storing and delivering all popular and infrequently accessed content and to handle our peak transcode loads. - Naren Venkataraman, Senior Director of Engineering, Vimeo

Fastly customers need low-latency, high-throughput storage and fast, flexible, secure content delivery at the edge. The combined power of Google Cloud Storage and Fastly’s Cloud Accelerator allows customers like Vimeo to fully optimize content storage and delivery, controlling costs and improving global performance. - Lee Chen, Head of Strategic Partnerships, FastlyGoogle Cloud Storage RegionalGoogle Compute EngineGoogle Cloud DataProcGoogle Cloud Machine LearningBigQuery

Spotify uses Google Cloud Storage for storing and serving music. Using Regional storage class allowed us to run audio transcoding in Google Compute Engine close to production storage. Google also offers great networking with open and explicit peering setup, as well as interconnect and partnerships with all of our CDN providers - Jyrki Pulliainen, Software Engineer, SpotifyAPI operations pricing

(click to enlarge)

New data management lifecycle capabilities data lifecycle management capabilities Expanding the Cloud Storage partner ecosystem

Fastly: Fastly is a content delivery network that lets businesses control how they serve content, provides real-time performance analytics and caches frequently changing content at the edge. Fastly enables customers to configure Google Cloud Storage as the origin, and Fastly’s Origin Shield designates a single point-of-presence (POP) to handle cache-misses across our entire network.

Veritas: Building on existing support for GCP, Veritas is committed to supporting Cloud Storage Coldline. The unique combination of Veritas Information Map, Veritas NetBackup and the GCP ensures customers can gain greater controls on data visibility as they move to the Google Cloud at global enterprise scale. Veritas' collaboration with Google further demonstrates the shared commitment to helping organizations around the world manage information.

Cloudian: The Cloudian HyperStore smart data storage platform seamlessly integrates with Google Cloud Storage (including Coldline) to provide anywhere from terabytes to hundreds of petabytes of on-premises storage. Policy-based data migration lets you move data to Coldline based on rules such as data type, age and frequency of access.

Cloudberry Lab: CloudBerry Backup is a cloud backup solution that leverages Coldline. In addition to offering real-time and/or scheduled regular backups, encryption, local disk image or bare metal restore, CloudBerry employs block level backup for maximum efficiency and provides alerting features to track each backup and restore plan remotely.

Komprise: Komprise data management software enables businesses to seamlessly manage the lifecycle of data and cut costs by over 70% by leveraging all the tiers of Cloud Storage transparently with existing on-premises storage. In under 15 minutes, customers can get a free assessment of how much data can move to Cloud Storage and the projected ROI with a free Komprise trial.

StorReduce: StorReduce’s inline deduplication software enables you to move terabytes to petabytes of data into Coldline (or other Cloud Storage tiers) and then use cloud services such as search on that data.

Cohesity: The Cohesity hyper-converged secondary storage system for enterprise data consolidates fragmented, inefficient islands of secondary storage into a virtually limitless storage platform. Coldline can be used for any data protection workload via. Cohesity’s policy-based administration capabilities.

Sureline: Sureline application mobility software delivers migration and recovery of virtual, cloud, physical or containerized applications and servers. It allows enterprises to use Coldline as the disaster recovery target for occasionally accessed DR images with SUREedge DR.

our web page hereour technical documentationsign up for a free trialcontact our sales team

Google Stackdriver is now generally available for hybrid cloud monitoring, logging and diagnostics

Thursday, October 20, 2016

Posted by Dan Belcher, Product Manager
Google StackdriverGoogle Cloud Platform1LoggingError ReportingTraceDebuggerintroduced a simple pricing modelsame infrastructure that powers the rest of Google

Uber uses Stackdriver Monitoring to monitor Google Compute Engine, Cloud VPN and other aspects of GCP. It uses Stackdriver alerts to notify on-call engineers when issues occur.

Khan Academy uses Stackdriver Monitoring dashboards to quickly identify issues within its online learning platform. It troubleshoots issues with our integrated Logging, Error Reporting and Tracing tools.

Wix uses Stackdriver Logging and Google BigQuery to analyze large volumes of logs from Compute Engine auto-scaled deployments. They get intelligence on system health state and error rates that provides essential insight for them to run their operations.

websitedocumentationsign up for a 30-day free trial
1 "Amazon Web Services" and "AWS" are trademarks of Amazon.com, Inc. or its affiliates in the United States

Introducing Pivotal Cloud Foundry on Google Cloud Platform

Wednesday, October 19, 2016

Posted by Jay Marshall, Principal Strategic AdvisorA new way for enterprises to capitalize on Google scale and innovationGoogle Cloud Platformbuild the most open cloudPivotalPivotal Cloud FoundryCloud Foundry

A differentiated Pivotal Cloud Foundry with Google

Cloud Foundry applications deployed on GCP utilize Google Cloud Load Balancing, which allows you to spin up from zero requests to millions automatically

Fast boot times on virtual machines for rapid scaling

Industry-leading price:performance ratio on cloud compute resources, including GCP-specific pricing features around per-minute billing, sustained use discounts and inferred usage discounts

custom-built service brokers

Google Cloud Vision API

Google Cloud Speech API

Google Cloud Natural Language API

Google Translate API

Customer collaboration - PCF and GCP in actionThe Home Depot Getting startedPivotal Cloud Foundry on Google Cloud Platform

What’s nextSpringcontact Sales

Introducing Google Cloud Shell’s new code editor

Tuesday, October 18, 2016

Posted by Sachin Kotwani, Product Manager
Google Cloud PlatformEclipse OrionGoogle Cloud Shell

Go to the Google Cloud Console

Click on the Cloud Shell icon on the top right section of the toolbar

Open the code editor from the Cloud Shell toolbar. You’ll also notice that we’ve introduced the ability to upload and download files from your Cloud Shell home directory.

Start editing your code and configuration files.

Cloud Shell code editor in actionGoogle Cloud Source RepositoryGoogle App EngineStackdriver Debugger
Create a sample app

On the Cloud Console website, select an existing project or create a new one from the toolbar.

Open Cloud Shell and the code editor as described above and create a new folder (File->New->Folder). Name it ‘helloworldapp’.

Inside the helloworldapp folder, create a new file and name it ‘app.yaml’. Paste the following:

runtime: python27
api_version: 1
threadsafe: yes

handlers:
- url: .*
  script: main.app

libraries:
- name: webapp2
  version: "2.5.2"

Create another file in the same directory, name it ‘main.py’, and paste the following:

#!/usr/bin/env python

import webapp2

class MainHandler(webapp2.RequestHandler):
    def get(self):
        self.response.write('Hello world!')

app = webapp2.WSGIApplication([
    ('/', MainHandler)
], debug=True)

Save your source code in Cloud Source Repositories

Switch to the tab with the open shell pane and go to your app’s directory:
cd helloworldapp

Initialize git and your repo. The first two steps aren't necessary if you've done them before:
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
git init
git add . -A
git commit -m "Initial commit"

Authorize Git to access GCP:
git config credential.helper gcloud.sh

Add the repository as a remote named ‘google’ to your local Git repository, first replacing [PROJECT_ID] with the name of your Cloud project:

git remote add google https://source.developers.google.com/p/[PROJECT_ID]/r/default

git push google master

Deploy to App Engine

From the ~/helloworldapp directory, type: gcloud app deploy app.yaml

Type ‘Y’ to confirm

Visit your newly deployed app at https://[PROJECT_ID].appspot.com

Use Stackdriver Debugger Debug page

Open main.py and click on a line number to set the debug snapshot location

Refresh the website displaying the hello world page, and you'll see the request snapshot taken in the debugger

Note that the Debug page displays the source code version of your deployed app

Summary—

Introducing Zaius, Google and Rackspace’s open server running IBM POWER9

Friday, October 14, 2016

Posted by John Zipfel, Technical Program Manager and Rob Lippert, Technical Lead, Zaius Program
OpenCAPI ConsortiumZaius P9 ServerOpen Server specificationBarreleye platforms

Block diagram of Zaius

The specificationsOpen BMCframework for whichOCP 2.0 mezzanine slot NIC

Managing containerized ASP.NET Core apps with Kubernetes

Thursday, October 13, 2016

Posted by Mete Atamel, Developer Advocate
Google Cloud Platform.NET Core

ASP.NET Core supports running apps across a variety of operating system platforms

KubernetesGoogle Container EngineASP.NET Core app, containerize it with Docker and and run it on Google App EngineGoogle Container Engine Prerequisites install .NET core, install Dockerinstall Google Cloud SDKcreate a Google Cloud Platform project Create a HelloWorld ASP.NET Core app

$ mkdir HelloWorld

$ cd HelloWorld

$ dotnet new -t web

$ dotnet restore

$ dotnet run

http://localhost:5000 Get the app ready for publishing

$ dotnet publish -c Release

$ cd bin/Release/netcoreapp1.1/publish/

$ dotnet HelloWorld.dll

Containerize the ASP.NET Core app with Docker

FROM microsoft/dotnet:1.1.0-runtime

COPY . /app

WORKDIR /app

EXPOSE 8080/tcp

ENV ASPNETCORE_URLS http://*:8080

ENTRYPOINT ["dotnet", "HelloWorld.dll"]

$ docker build -t gcr.io/<PROJECT_ID>/hello-dotnet:v1 .

$ docker run -d -p 8080:8080 -t gcr.io/<PROJECT_ID>/hello-dotnet:v1

http://localhost:8080 Create a Kubernetes cluster in Container Enginekubectl

$ gcloud components install kubectl

kubectl

$ gcloud container clusters get-credentials hello-dotnet-cluster \

--zone europe-west1-b --project <PROJECT_ID>

Now, let’s push our image to Google Container Registry using gcloud, so we can later refer to this image when we deploy and run our Kubernetes cluster. In the Google Cloud SDK Shell, type:

$ gcloud docker push gcr.io/<PROJECT_ID>/hello-dotnet:v1

Create a Kubernetes cluster with two nodes in Container Engine:

$ gcloud container clusters create hello-dotnet-cluster --num-nodes 2 --machine-type n1-standard-1

This will take a little while but when the cluster is ready, you should see something like this:

Creating cluster hello-dotnet-cluster...done.

Deploy and run the app in Container Enginekubectl

$ kubectl run hello-dotnet --image=gcr.io/<PROJECT_ID>hello-dotnet:v1 \

--port=8080

deployment “hello-dotnet” created

Make sure the deployment and pod are running:

$ kubectl get deployments

NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE

hello-dotnet 1 1 1 0 28s

$ kubectl get pods

NAME READY STATUS RESTARTS AGE

hello-dotnet-3797665162-gu99e 1/1 Running 0 1m

And expose our deployment to the outside world:

$ kubectl expose deployment hello-dotnet --type="LoadBalancer"

service "hello-dotnet" exposed

Once the service is ready, we can see the external IP address:

$ kubectl get services

NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE

hello-dotnet XX.X.XXX.XXX XXX.XXX.XX.XXX 8080/TCP 1m

Finally, if you visit the external IP address on port 8080, you should see the default ASP.NET Core app managed by Kubernetes!

It’s fantastic to see the ASP.NET and Linux worlds are coming together. With Kubernetes, ASP.NET Core apps can benefit from automated deployments, scaling, reliable upgrades and much more. It’s a great time to be a .NET developer, for sure!