SOA Series Part 4: Caching Service Responses Client-Side

This is the fourth in a series of seven posts on service-oriented architecture derived from a workshop conducted by Cloves Carneiro and Tim Schmelmer at Abril Pro Ruby. The series is called SOA from Day One – A love story in 7 parts.

Service calls – when a client makes a request to a service (using a pre-defined protocol and API) and the service delivers back its response to the client – can be expensive for your application to make. In this part of the “SOA from Day One” series of blog posts, we will be looking at ways to mitigate the costs of making such calls by caching service results client-side.

Don’t even make a service call

Fortunately, service clients can often tolerate data that is slightly stale. A good way to exploit this fact is to cache service responses within the client application (which can of course be both consumer facing front-end applications, as well as other services that depend on data from your service). If, for example, the client can tolerate service data being 5 minutes old, then it will incur the inter-application communication costs only once every 5 minutes. The general rule here is that the fastest service calls are the ones your applications never need to make.

The rest of this article lays out a few things we learned when considering these points:

Build caching into your client gem

Often the team that builds a service will also be developing the libraries (e.g., Ruby client gems) to access that service. This is at least the way we develop client libraries at LivingSocial, and we build them on top of a general low-level gem that takes care of all general HTTP communication and JSON (de)serialization.

Some of these client gems are built to offer the option to have a cache store object injected into them via a configuration option. A thusly injected cache store can be used to store all response objects returned by the service.

NOTE: it is important to think about the construction of the key by which such objects are cached. These keys need to be unique, and should most likely at least include the gem and service API versions, as well as a unique key for the service API called. Here is an example to illustrate some cache-key calculating code we use in one of our gems:

1
2
3
4
5
6
7
8
9
10
11
def self.key_for_foo_api(foo_id, opts)
  key_with("foo_api:#{opts.hash}", foo_id)
end

def self.key_with(dynamic_part, id = nil)
  [common_key, dynamic_part, id].join(':')
end

def self.common_key
  ["ls-foo-gem", LS::Foo::VERSION, LS::Foo.foo_service_host, LS::Foo.cache_version]
end

A prudent approach when allowing cache store injection in your client library is to require as little as feasibly possible about the cache object that is injected. That strategy provides the client applications with more flexibility around choosing which cache store to use.

The following sample code of a cache helper is used in one of our client gems:

1
2
3
4
5
6
7
8
9
def with_possible_cache(cache_key)
  if cache
    cache.fetch(cache_key) do
      yield
    end
  else
    yield
  end
end

Note that all this code requires of the cache object is that it have a #fetch method that takes a cache key and a block to execute on cache miss.

The client gem then uses this method to wrap all code that calls out to the service fronted by this client gem, for example in the fictitious find_by_name API to retrieve details of a named resource:

1
2
3
4
5
6
7
def find_by_name(name)
  with_possible_cache(cache_key_for('find_by_name', name)) do
    Hashie::Mash.new(
      JSON.parse(Typhoeus.get("#{host}/api/v#{api_version}/#{resources}/#{name}").body)
    )
  end
end

Cache details are a client concern

Caching should be entirely a client application’s responsibility. Some clients might not want to cache at all, so make sure to provide the ability to just disable caching in your client gem.

It is also the client applications’ responsibility to set cache expiration policies, cache size, and cache store back-end. As an example, Rails client applications could simply use Rails.cache, some others might prefer a Dalli::Client instance (backed by memcached), an ActiveSupport::Cache::MemoryStore (backed by in-process RAM), or even something that is entirely ‘hand-rolled’ by your clients, as long as it conforms to the protocol your gem code expects of the cache store.

As always when attempting performance improvements, make sure to benchmark before picking a caching approach. E.g., long-running (rarely deployed) applications might perform better when using an in-memory cache, while other applications profit more from the synergy of sharing cache data in a remote / out-of-client-process cache store, like for example memcached.

Exercise “Client-side caching”

In this exercise you will build a ‘client-gem like’ code (albeit inside of the inventory_service application), which will allow for caching; you will also set up your inventory_service code to administer a cache for this code.

  • Abstract common functionality of the RemoteTag and RemoteCity classes into a common superclass (e.g., named RemoteModel)
    • add an accessor for RemoteModel.cache, so the application code can choose the cache store to use
    • implement a RemoteModel.with_possible_cache(cache_key, &blk), and wrap all calls to cities_service and tags_service inside it.
    • make sure the cache_key passed takes the full service call URL and the API version into account
  • Add administration of the cache inside of the tags_service.rb and cities_service.rb initializers in inventory_service. You can choose whichever cache store you like. (We chose ActiveSupport::Cache::MemoryStore for simplicity)

Tip: Solutions to this exercise can be found in this branch of the inventory_service application

Previous articles in this series:

Comments