Google Cloud Platform Blog: Tableau, Google BigQuery, & Twitter - Visualization of Streamed Tweets at #GCPNext

Tableau, Google BigQuery, & Twitter - Visualization of Streamed Tweets at #GCPNext

Wednesday, July 29, 2015

Today's guest post comes from our friends at Tableau: Jeff Feng, Product Manager & Ellie Fields, Vice President of Product Marketing. Tableau, a Google Cloud Platform partner, is a leader of interactive data visualization software. “It’s a beautiful thing when best-of-breed technologies — Tableau, Google BigQuery and Twitter — come together to operate seamlessly in concert with one another.” - Jeff Feng Next, a Google Cloud Platform Series Over the month of June, the Tableau team traveled around the world with the Google Cloud Platform team as a proud sponsor of Next, a Google Cloud Platform event series. The teams made stops in New York, San Francisco, Tokyo, London, and Amsterdam where attendees learned about the latest services and features on the platform, and fellow developers and IT professionals shared how they are using Google Cloud Platform to move from idea to an application and/or decision quickly.

Ellie presented a joint demo on Twitter during the Data & Analytics Talk at Next, New York City (Left). Jeff discussed the activity of Tweets around #GCPNext in Amsterdam (Right). Visualizing Streamed Tweets with Tableau, Google BigQuery & Twitter As a part of our presence at the events, we wanted to develop a live demo that highlighted and showcased our technologies. Google BigQuery has the ability to process petabytes of data within seconds and ingest data rapidly. Tableau’s live connectivity to BigQuery enables users to create stunning dashboards within minutes with our drag-and-drop interface, extending the usefulness of BigQuery to all users. For this demo, we decided to visualize real-time Tweets from Twitter about the #GCPNext conference series.

Overall architecture for visualizing streamed Tweets in BigQuery using Tableau. We worked together with our friends at Twitter (@TwitterDev) who developed an open-source connector called Twitter-for-BigQuery that streams Tweets directly into BigQuery. Additionally, the connector can retrieve the last 30 days of data for the defined Tweet stream. The APIs for the connector are provided by Gnip, which offers enterprise-grade access and filtering for the full Twitter stream. The connector enables users to define the filters for certain hashtags and usernames, and consequently streams tweets matching these filters in real time directly into BigQuery using the Tabledata.insertAll method. For the purposes of our demo, our Tweet stream included hashtags such as #bigdata, #IoT, and #GCPNext as well as usernames such as @Google. Once the data lands in BigQuery’s tables, the data may be accessed using super-fast, SQL-like queries using the processing power of Google’s infrastructure. Google provides a console with a command line interface that’s great for analysts and developers who know how to write SQL. Tableau enhances the joint solution by providing a drag-and-drop visual interface to the data so that anybody can use it. Plus our live native connector to Google using the BigQuery REST API means a user can leverage our interface while optimized against Google’s massive infrastructure. Additionally, Tableau and the Google BigQuery team have co-published a best practices whitepaper to help you maximize the value of our joint solution. Using Tableau Desktop, we connected to the data and built the dashboard below, enabling users to search for keywords within the filtered Tweet stream. Then we published the live data connection to BigQuery and the dashboard to Tableau Online, our hosted analytics platform. Tableau Online is the perfect compliment to BigQuery because the solution is completely no-Ops and maintenance-free. It also supports a live connection to Google BigQuery. Not only does the dashboard show the overall number of Tweets in the stream and the percentage occurrence of the keyword by date, but you can also visualize the actual Tweet itself by hovering over the marks in the scatter plot below.

Interactive Tableau Online dashboard visualizing live streamed Tweets in Google BigQuery.

In the video below, Ellie shares how you can interact with the Tableau Online visualization we created as well as build a new visualization using the live data connection to BigQuery directly from Tableau Online.

Demo video featuring Tableau Online visualizing live streamed Tweets in Google BigQuery. What’s Up Next? At Tableau, we believe that the future of data is in the cloud. We love how Google is innovating on cloud infrastructure and building the cloud services of tomorrow today. That’s why we recently announced a new named connector to Google Cloud SQL. The connector moves Google Cloud Platform and Tableau Online customers one step closer to being able to both host and analyze data completely in the cloud. This connector also compliments our existing native connectors to Google BigQuery and Google Analytics. In the future, we are committed to building broader and deeper integrations with Google to delight our users. Try It For Yourself! The beautiful thing about this demo is that the technologies used in the solution are easy to use. To learn more and try it for yourself, please see the following links below: