Cloud Bigtable + Cloud Memorystore: faster togetherCloud Bigtable + Cloud Memorystore: faster togetherDeveloper Relations Engineer, Google Cloud

Databases are designed for specific schemas, queries, and throughput, but if you have data that gets queried more frequently for a period of time, you may want to reduce the load on your database by introducing a cache layer. 

In this post, we’ll look at the horizontally scalable Google Cloud Bigtable, which is great for high-throughput reads and writes. Performance can be optimized by ensuring rows are queried somewhat uniformly across the database. If we introduce a cache for more frequently queried rows, we speed up our application in two ways: we are reducing the load on hotspotted rows and speeding up responses by regionally colocating the cache and computing. 

Memcached is an in-memory key-value store for small chunks of arbitrary data, and I’m going to use the scalable, fully managed Memorystore for Memcached, since it is well integrated with the Google Cloud ecosystem.

Setup

  1. Create a new Google Cloud project or use an existing project and database of your choice. The examples here will show Cloud Bigtable, but Spanner or Firestore would be good options too.

  2. I’ll provide gcloud commands for most of the steps, but you can do most of this in the Google Cloud Console if you prefer.

  3. Create a Cloud Bigtable instance and a table with one row using these commands:

cbt createinstance bt-cache “Bigtable with cache” bt-cache-c1 us-central1-b 1 SSD && \ 

cbt -instance=bt-cache createtable mobile-time-series “families=stats_summary” && \ 

cbt -instance=bt-cache set mobile-time-series phone#4c410523#20190501 stats_summary:os_build=PQ2A.190405.003 stats_summary:os_name=android && \ 

cbt -instance=bt-cache read mobile-time-series

The code

The generic logic for a cache can be defined in the following steps: 

  1. Pick a row key to query.

  2. If row key is in cache

  • Return the value.

     3. Otherwise

  • Look up the row in Cloud Bigtable.
  • Add the value to the cache with an expiration.
  • Return the value.

For Cloud Bigtable, your code might look like this (full code on GitHub):