decoupling callback & scheduling

From: Robert Collins <robertc@dont-contact.us>
Date: Tue, 24 Aug 2004 11:31:10 +1000

in squid 2 & 3 we use the following general approach for callbacks:

// lock callback data
// call callback based routine
someObject.someRoutine(parameters, cbdata, callbackroutine)
return

someRoutine then does:
// save callbackdata
self->callback=callbackroutine
self->cbdata=cbdata
// kick off async activity

and eventually:
// call callback
callback=self->callback
cbdata=self->cbdata
callback(cbdata, someresult,...)
return

I've been playing with Twisted (www.twistedmatrix.com) for a while, as
part of a work project. other than language it greatly resembles squid.
(single threaded, async, callback..)

One interesting pattern they have is the 'deferred' pattern.
http://www.twistedmatrix.com/documents/current/api/twisted.internet.defer.html
http://www.twistedmatrix.com/documents/howto/defer

I'd like to suggest that we adopt this for squid 3.1 development.

What would it do for us?

*) the Deferred approach decouples success & failure handling for
callbacks, leading to cleaner code.
*) the api for calling callback routines decouples the generation of
results, and the calling of the recievers - rather than passing in where
to call, and with what, a callback routine just returns a deferred
instance that represents what it will generate later. This allows
runtime decisions to (for instance) process io data on the next event
loop, or immediately on the callstack.
*) common idioms for callbacks can be centralised and reused.

roughly, using a deferred is done as follows

deferred=someObject.someRoutine(parameters)
deferred.addCallback(method, self) // locking done for us
deferred.addErrback(method, self) // ditto
return

someRoutine then does:
// allocate a deferred
self.deferred=Deferred<type signature>()
// kick off async activity
return self.deferred

and eventually:
// call callback
self.deferred.callback(result)
//or
self.deferred.errback(Failure)
return

Unless we use RTTI and a common base type, we'll need to use a template
Deferred as I show above, sacrificing some of the flexability of the
Deferred that twisted has - but I don't think that that will be a
significant issue. Regardless, I'd write up a Deferred example set of
tests and uses to see how it fits with what we do, before incorporating
it into squid.

The real question is - what do y'all think?

Rob

Received on Mon Aug 23 2004 - 19:31:26 MDT

This archive was generated by hypermail pre-2.1.9 : Wed Sep 01 2004 - 12:00:04 MDT