Google Cloud Platform Blog
Product updates, customer stories, and tips and tricks on Google Cloud Platform
Offline Disk Import and the OmNomNom Machine
Thursday, November 14, 2013
Yesterday, we announced that we are
expanding our offline disk import service
to better serve users globally. With new disk upload centers in Switzerland, Japan and India, as well as our US center, it’s easier for people around the world to import large data sets by mailing hard drives to us rather than sending hundreds of terabytes over their slow, expensive or unreliable Internet connection.
But importing large amounts of data at scale isn’t simple. Our engineers have been working on the challenge for years. Originally, offline disk import was handled at our
data centers
as a way to efficiently import large amounts of data from the hard drives in our
Street View
cars - vehicles that capture terabytes of photographs and information about the landscape as they build a navigable, visual database of the world.
And although it’s technically challenging, the system we built for rapidly ingesting and processing these large data sources has a playful name. We call it
OmNomNom
. Here is one of the test OmNomNom machines that the disk import team has at its offices in Mountain View:
But rapidly ingesting and processing large data sets and making them usefully available is a bit more complex than gobbling down a cookie. As we’ve improved our ability to quickly import these drives, we dramatically reduced the time between capturing these images and making them available to users around the world.
Now, we are helping people across the world take advantage of the speed, scale and global availability of Google Cloud Storage as well as this rapid disk-upload technology. Even though it might sound like something out of Sesame Street, this is another example of how Google Cloud Platform is making the advantages of Google-sized scalable infrastructure available to you. All you need to do is send us your EncFS encrypted hard drives, and we will let you know once your encrypted bytes are imported to your designated GCS bucket. Once uploaded, we can mail your drives back to you, or if you prefer, safely and securely handle disk destruction free of charge. Check out our website for how you can be part of the
Limited Preview of International Offline Disk Import
.
(Oh, and for those of you who want to use Google Cloud Storage but don’t need offline disk import, you always have quick access Google Cloud Storage from the command line using
gsutil
).
-Posted by Benjamin Bechtolsheim, Product Marketing Manager
Free Trial
GCP Blogs
Big Data & Machine Learning
Kubernetes
GCP Japan Blog
Firebase Blog
Apigee Blog
Popular Posts
Understanding Cloud Pricing
World's largest event dataset now publicly available in BigQuery
A look inside Google’s Data Center Networks
Enter the Andromeda zone - Google Cloud Platform’s latest networking stack
Getting your data on, and off, of Google App Engine
Labels
Announcements
193
Big Data & Machine Learning
134
Compute
271
Containers & Kubernetes
92
CRE
27
Customers
107
Developer Tools & Insights
151
Events
38
Infrastructure
44
Management Tools
87
Networking
43
Open
1
Open Source
135
Partners
102
Pricing
28
Security & Identity
85
Solutions
24
Stackdriver
24
Storage & Databases
164
Weekly Roundups
20
Feed
Subscribe by email
Demonstrate your proficiency to design, build and manage solutions on Google Cloud Platform.
Learn More
Technical questions? Check us out on
Stack Overflow
.
Subscribe to
our monthly newsletter
.
Google
on
Follow @googlecloud
Follow
Follow
Follow