Redefining CMS efficiency with Memcached

Let's talk about Memcached. In the modern web at scale, we are presented with a problem. In order to deal with traffic volume and diurnal traffic volume changes, the most economical way is to set up an load balancer with fleet of smaller virtual servers like this:

The load balancer is only responsible for two things: distributing traffic equally among the servers, and also creating new servers if the traffic exceeds their thresholds and removing excess servers below a certain threshold. It's called horizontal scaling. It works great, however in the case of a CMS we have another problem. Should each of those servers have to make the same requests to the database in parallel? They would without any special precaution to prevent it.

Our solution was to give them a connection to a memcache cluster, so that they all have a type of RAM that's available to each of them. It's shared. Memcache is just a hashed object-store. Generally those objects are under 1MB, and they are accessible FAST. We are talking 5 µs or less, lightspeed from one side of the data center to another. This is particularly handy because now we have a way to sync all the servers to the same state, within just a few microseconds of each other. It means that they don't have to make the same DB request in parallel, and that their object will all expire about the same time. Whoever needs it first will re-request it from itself (via the DB, slow, like 20-100ms), and as soon as the response arrives it gets set back into the memcache, available for everybody.

These responses are handled as HTTP requests in our architecture, because content is retrieved from arbitrary data feeds. They can be requesting from Viking CMS (itself), or from any other endpoint. We don't make a distinction where it comes from. The response data gets cached in the memcache the same for any data feed!

This means that we have the best of both worlds, high availability, horizontal scalability extremely low latency, and fast TTL on content changes. Nobody else offers that.