The real-time web is increasingly all around us – tracking your ride approaching on a map while you wait, seeing up-to-date statistics on a live dashboard, getting fresh messages about your friends’ social activities. These are all ways in which we’ve come to expect our online experience to reflect reality in real-time. Bringing users timely insight requires more sophisticated handling of data – data that's being generated and processed in ever-increasing volume and diversity.

By real-time, we don’t mean it in the strict sense used by hardware controllers, but near real-time information generally perceived to be more or less immediate, useful within seconds rather than minutes. The diversity of solutions available to address real-time oriented problems is great. Many tools exist to help tackle different pieces of the real-time application puzzle. Google Cloud Platform provides a wide range of components to help you build real-time oriented solutions. Examples include high performance infrastructure, such as Google Compute Engine virtual machines, advanced container capabilities in the form of Google Container Engine, stream-oriented big data services, as well as a fully managed database oriented toward real-time applications, Firebase.

We’ve recently documented two tutorials demonstrating how you can compose platform services into specific solutions for real-time oriented problems. Both leverage the power of Google BigQuery to perform fast, insight-yielding queries on large amounts of data that grow and update in real-time relative to their sources. BigQuery supports a streaming option for loading data and these two initial solutions both leverage that API.

The first tutorial outlines how to perform sentiment analysis on a Twitter stream using BigQuery. To handle the high volume intake, the tutorial demonstrates how to use Kubernetes-managed containers to build an asynchronous, real-time pipeline that buffers Twitter data in Redis before streaming into BigQuery. The second tutorial demonstrates how to use the log forwarding tool Fluentd with a BigQuery connector to stream log data into BigQuery. This data can then be visualized with a chart directly in Google Spreadsheets with native Apps Script support for BigQuery.

On the mobile front, Firebase is a managed database with a powerful, developer-friendly API to store and sync data in real-time. Mobile and web client side SDKs automatically sync local data state whenever the data is changed in the database. Particularly useful for mobile developers is the fact that this synchronization is completely tolerant of offline local data changes – even when users go into airplane mode or descend into the subway.

This is just a glimpse into the myriad of ways in which tomorrow's real-time problems can be solved today on Google Cloud Platform. We'll continue to add other real-time oriented solutions and information to the section on solutions for real-time.

-Posted by Preston Holmes, Cloud Solutions Architect