MintCache for Django

posted: 2007-03-17 02:21:39 perma-link, RSS comments feed

MintCache is a caching engine for django that allows you to get by with stale data while you freshen your breath, so to speak.

The purpose of this caching scheme is to avoid the dog-pile effect. Dog-piling is what normally happens when your data for the cache takes more time to generate than your server is answering requests per second. In other words if your data takes 5 seconds to generate and you are serving 10 requests per second, then when the data expires the normal cache schemes will spawn 50 attempts at regenerating the data before the first request completes. The increased load from the 49 redundant processes may further increase the time it takes to generate the data. If this happens then you are well on your way into a death spiral.

MintCache works to prevent this scenario by using memcached to to keep track of not just an expiration date, but also a stale date. The first client to request data past the stale date is asked to refresh the data, while subsequent requests are given the stale but not-yet-expired data as if it were fresh, with the undertanding that it will get refreshed in a 'reasonable' amount of time by that initial request.

I don't think django has a mechanism for registering alternative cache engines, or if it does I jumped past it somehow. Here's an excerpt from my cache.py where I've just added it alongside the existing code. You'll have to hook it in yourself for the time being. ;-(

-------------- the code ---------------
try:
import memcache
except ImportError:
_MintCache = None
else:
class _MintCache(_Cache):
"Memcached cache backend the sequel."
def __init__(self, server, params):
_Cache.__init__(self, params)
self._cache = memcache.Client(server.split(';'))
def get(self, key, default=None):
key = self.scrub_key( key )
val = self._cache.get(key)
if val is None:
val = default
else:
try:
stale_after,val = pickle.loads(val)
now = time.time()
if now > stale_after:
cache_log( "stale, refreshing" )
self.set( key, val, 60 )
# the old val will now last for 60 additional secs
val = default
except:
pass
return val
def set(self, key, value, timeout=0):
key = self.scrub_key( key )
if timeout is 0:
timeout = self.default_timeout
now = time.time()
val = pickle.dumps( ( now + timeout, value ), 2)
self._cache.set(key, val, 2 * timeout ) # 7*86400)
def delete(self, key):
key = self.scrub_key( key )
self._cache.delete(key)
-------------- /the code ---------------


I've spent a bit of time thinking about how caching works in django ( in particular 0.90 ) for work, and I've employed a variation on this that leverages memcache and disk caching in concert. Both of these approaches are improvements over the existing cache paradigm for django that don't require any code changes.

But I do have schemes that are a little more revolutionary than evolutionary. The first changes the API in such a way as to limit the entire API to a single call -- get. In conventional cache usage you might see code like this:

key = _generate_cache_key()
if ! cache.has_key( key ):
val = do_some_work()
cache.set( key, val )
else:
val = cache.get( key )


For that the programmer ends up using .get, .set, and .has_key. I elliminate 2/3 of the api and 2/3 of the code with a revised api like so:

key = _generate_cache_key()
val = cache.get( key, do_some_work )


Here, rather than checking the cache and calling do_some_work when needed, I pass a callable ( could be a function, method, lambda, closure,... ) into get, and the cache engine gets to decide what to do. It t could return a cached value, it could block and return the value of do_some_work() after caching it. But where it get realy interesting ( at least to me ) is that it could return a stale value immediately after putting the callback in a queue that is being processed in a separate thread. In that case, no one has to pay the penalty for asking for expired content. The refresh occurs out of band.

This is the kind of stuff I really like, and you see a lot of it in python. Not only did our code get shorter, but it got more readable, it became simpler to write, and it got more powerful.

---

Comments

Post a comment

Your name:

Comment: