Re: scalable event scheduling

On 01/30/2013 07:12 PM, Rainer Weikusat wrote:
> Alex Rousskov <rousskov_at_measurement-factory.com> writes:
>> FWIW, here is how I think the event cancellation should be supported
>> instead:

>> 1. Another form of eventAdd() should be added to accept caller's async
>> call. New code, especially async jobs, should use that form (jobs will
>> and avoid call wrappers that way). Old code that wants to cancel events
>> should use this new form as well.

> This would imply keeping cancelled events on the event scheduler queue
> until they expire 'on their own' because the metainformation necessary
> to remove them efficiently from this queue is not available in this
> way.

The API I sketched does not preclude supporting efficient removal of
canceled events from the queue. Those are two separate issues! For
example, the event queue code could efficiently map the supplied async
call pointer to the event when deleteEvent() is called. Using that event
pointer, the event can be deleted efficiently.

> A better idea would be to add an interface which takes a call instead
> of the function pointer + argument pointer as argument

That is exactly what step #1 in my sketch (quoted above) describes.

> so that all
> options would be available to be used whenever their use happens to
> make sense (an even better idea would be to integrate these two
> separate things in some 'closure-like' object and to relieve users
> from the burden of worrying about two pointers instead of one but
> that's sort of a digression).

I am not sure what you mean by "options" but async calls are
closure-like objects, essentially.

>> The current event scheduler already creates an async call object for
>> nearly all events anyway, so this does not add CPU overhead. The only
>> overhead is that we will consume the same [small] amount of RAM
>> sooner.
>
> It actually does this for all events.

No, it does not do that for events removed from the event queue. While
that rarely happens, my sketch will add an overhead in those cases (but
so will any scheme that allocates another handle for the event).

> This kind of double-indirection
> is what makes getting rid of an even which is possibly still on the
> event-scheduler queue so difficult

I do not see the difficulty: the current code already supports removal
from the event queue (it just needs some polishing to carefully relax
"must be here" assertions). And if the caller has the async call, then
dealing with the events already in the async queue stops being a problem.

> Unless there's (again) something I'm missing, running event
> scheduler events directly from the EventScheduler::checkEvents routine
> should be considered as 'general case'. 'Running async calls' could be
> provided on top of that using a suitable, general 'event running
> routine'.

I am not sure I understand the above, so please do not interpret the
comments below as a criticism of some sort, just an explanation of why
events are supported using async queue and not the other way around:

* Early async calls were implemented via events. We consciously went
away from that model.

* Most events are rare and deal with performance-insensitive code. They
are for the code that wants (and can afford) to wait. Nearly all events
with zero timeouts have been or should be replaced with async calls(**).

* Async queue is the opposite of the event queue as far as performance
is concerned: It is in the middle of most performance-sensitive
processing and, hence, needs to be as fast as possible.

What is currently missing is async call support in the event queue,
adding job protection and reliable event cancellation. It is quite easy
to add as sketched in the previous email. Implementing that is a nice (a
little challenging but compact and generally useful) project looking for
a volunteer.

HTH,

Alex.
(**)With an important (but rare) exception of so called "heavy events".
Those heavy events can afford to wait a little as well though.
Received on Thu Jan 31 2013 - 04:56:10 MST