Google Cloud Platform Blog: Leverage the power of log data to drive operational and business decisions through Google Cloud Logging

Google Cloud Platform Blog

Product updates, customer stories, and tips and tricks on Google Cloud Platform

Leverage the power of log data to drive operational and business decisions through Google Cloud Logging

2015年3月19日星期四

Google Cloud LoggingGoogle Compute EngineGoogle App EngineGoogle Cloud Monitoring

Ingest and view the log data, so that you can see all your logs in one place

Search the log data in real-time, so that you can resolve operational issues

Analyze the log data in real-time, so that you can glean actionable insights

Archive logs data for longer periods, to meet backup and compliance requirements

At Wix we use BigQuery to analyze logs of Compute Engine auto-scaled deployments. We get a large volume of syslog data that we send to BigQuery to get insights on system health state and error rates. We generate time series data and integrate it with Google Cloud Monitoring to monitor system performance and business metrics. This provides us with essential insight for the running of our operations. - Dmitry Shestak, Engineer@Infrastructure team, Wix Ingest and view the log data

Compute Engine VM logs can be automatically collected for about two dozen log types through the Google packaged fluentd agent, with additional logs possible through custom configuration.

Compute Engine Activity logs record all system actions and API calls are enabled by default, with no agent installation required.

App Engine logs that include syslog, request logs and application logs are automatically enabled for all App Engine projects, including applications using Managed VM runtimes.

Google Developers Console

When viewing logs in the Logs Viewer, you can filter results using filter text or drop-downs

Search the log data in real-time

A filtered view of logs data using the Logs Viewer

Analyze the log data in real-timeGoogle BigQueryExportsdetailed documentation

Log data in the BigQuery tables

Monitoring Code Performance: There are situations when something unexpected happened or is indicative of an imminent problem e.g. “disk space low.” With Compute Engine log data in BigQuery, you can generate a time series and monitor logs with a particular severity. It’s simple, you just query metadata.severity = “WARNING” in the relevant tables. E.g.

SELECT Count (*) AS total, Date(metadata.timestamp) AS time FROM   (Table_date_range(TABLE ID,  Timestamp('2015-03-01'), Timestamp('2015-03-12')))  WHERE  metadata.severity = "warning"  GROUP  BY time ORDER  BY total;

Monitoring Request Latency: High latency leads to poor user experience and failed requests, which can lead to frustrated users and lost revenue. With App Engine log data in BigQuery, you can create time series of latency data by aggregating and charting the “protoPayload.latency” field. You can see unusual latencies in real-time and take steps to resolve the issue.

Archive logs data for longer period Google Cloud StorageGoogle Cloud DataflowHadoop Google Cloud Storage Nearlinemore affordable Getting StartedCloud Logging documentation pagefeedback

Free Trial

GCP Blogs

Big Data & Machine Learning
Kubernetes
GCP Japan Blog
Firebase Blog
Apigee Blog

Popular Posts

World's largest event dataset now publicly available in BigQuery
Understanding Cloud Pricing
A look inside Google’s Data Center Networks
Enter the Andromeda zone - Google Cloud Platform’s latest networking stack
Getting your data on, and off, of Google App Engine

标签

Announcements 193
Big Data & Machine Learning 134
Compute 271
Containers & Kubernetes 92
CRE 27
Customers 107
Developer Tools & Insights 151
Events 38
Infrastructure 44
Management Tools 87
Networking 43
Open 1
Open Source 135
Partners 102
Pricing 28
Security & Identity 85
Solutions 24
Stackdriver 24
Storage & Databases 164
Weekly Roundups 20

Feed

Subscribe by email

Demonstrate your proficiency to design, build and manage solutions on Google Cloud Platform.

Technical questions? Check us out on Stack Overflow.
Subscribe to our monthly newsletter.

Company-wide

Official Google Blog
Enterprise Blog
Student Blog

Products

Official Android Blog
Chrome Blog
Lat Long Blog

Developers

Ads Developer Blog
Android Developers Blog
Developers Blog

Google
Privacy
Terms