Jump to Content
Google Cloud

OpenStack users: Back up your Cinder volumes to Google Cloud Storage

April 11, 2016
Ben Chong

Product Manager, Transfer Appliance

Mark Lambert

Google Cloud Platform, Google Cloud Platform

OpenStack Mitaka has just launched and we’re super excited about it. In collaboration with Red Hat and Biarca, we’ve developed an OpenStack Cinder backup driver for Google Cloud Storage, available in the Mitaka release.

https://storage.googleapis.com/gweb-cloudblog-publish/images/mitakau0d9.max-400x400.PNG

Google joined the OpenStack Foundation in July 2015, when we announced Kubernetes integration with OpenStack. Our work on Mitaka is the next step on our roadmap to making Google Cloud Platform a seamless public cloud complement for OpenStack environments.

Backup and recovery services represent one of the most costly and complex aspects of large scale infrastructure management. OpenStack provides an efficient mechanism for allocation and management of persistent block storage through Cinder. In an OpenStack deployment, Cinder volumes house virtual machine data at rest as well as, potentially, the operating system boot device. In production deployments, it’s critical that this persistent data is protected as part of a comprehensive business continuity and disaster recovery strategy. To satisfy this requirement, Cinder provides a backup service that includes a backup driver specification allowing storage vendors to add support for additional backup targets.

This is where we come in. The addition of highly durable and available cloud-scale object storage allows organizations to shift from bulk commodity storage for backup to a more operationally efficient and cost-effective architecture, all while avoiding additional capital expenditures and the complexity of managing storage device scale out. The traditional barrier to adoption for object storage is the engineering effort required to adapt existing software and systems, designed for either file or block storage access, to object store native REST interfaces. The Cinder backup driver model provides the potential to abstract this engineering complexity for OpenStack users. As long as an appropriate backup driver is installed, the backup target works with Cinder as intended.

Our OpenStack Cinder backup driver is included as part of the standard Cinder backup driver set in Mitaka and requires minimal setup to get up and running. Full Cinder backup functionality was successfully tested with the Cloud Storage driver against 1GB, 5GB and 10GB Cinder volume sizes. In addition, the driver provides the following user configurable parameters to allow administrators to tune the installation:

ParameterPurpose
backup_gcs_credential_file Denotes the full path of the json file of the Google service account (downloaded from the Google Developer Console in step 3)
backup_gcs_bucket GCS bucket name to use for backup. Please refer to the official bucket naming guidelines.
backup_gcs_driverUsed for selecting the Google backup driver
backup_gcs_project_id Denotes the project ID where the backup bucket will be created
backup_gcs_object_size The size in bytes of GCS backup objects. default: 52428800 bytes
backup_gcs_block_size The change tracking size for incremental backup, in bytes. backup_gcs_object_size has to be a multiple of backup_gcs_block_size default: 327678 bytes
backup_gcs_user_agenthttp user-agent string for the gcs API
backup_gcs_reader_chunk_sizeChunk size for GCS object downloads in bytes. default: 2097152 bytes
backup_gcs_writer_chunk_size Chunk size for GCS object uploads in bytes. Pass in a value of -1 to cause the file to be uploaded as a single chunk. default: 2097152 bytes
backup_gcs_num_retries/td>Number of times to retry transfers. default: 3
backup_gcs_bucket_locationLocation of GCS bucket. default: ‘US’
backup_gcs_storage_classStorage class of GCS bucket. default: ‘NEARLINE’
backup_gcs_retry_error_codesList of GCS error codes for which to initiate a retry. default: [‘429’]
backup_gcs_enable_progress_timerEnable or Disable the timer to send the periodic progress notifications to Ceilometer when backing up the volume to the GCS backend storage. The default value is True to enable the timer. default: True

The Cinder backup driver works with any class of Cloud Storage, including our Google Cloud Storage Nearline or Coldline archival options. Google Cloud’s archival storage options offer the same durability and low latency of hot storage. The low cost yet high performance of Nearline and Coldline makes backing up Cinder volumes economical while offering the ability to quickly restore if necessary.

If you’re running OpenStack, there’s no need to invest in additional storage systems or build out a second data center for backup and recovery. You can now use Cloud Storage in a hybrid scenario, optimized via the Cinder backup driver available in Mitaka.

Editor’s note: This post was updated in October 2018 to reflect updated Cloud Storage capacity information.

Posted in