Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

From: Henrik Nordström <henrik_at_henriknordstrom.net>
Date: Tue, 04 Dec 2012 00:16:29 +0100

mån 2012-12-03 klockan 15:53 -0700 skrev Alex Rousskov:

> True (although the current small maximum select(2) delay is a result of
> bugs elsewhere in the code and should not really be there). If we can do
> the right thing here easily, we should (instead of adding more future
> problems). And I think we can.

Signal processing is however always due to these races, unless using a
comm pipe to wake up the main loop on receiving a signal.

> Ah, I see another problem with the "each non-waiting engine runs once"
> approach. One engine may create work for other engines or for itself,
> including async calls and "run this now" lightweight events. This means
> we should really continue to use the sawActivity loop so that all
> lightweight events are processed before I/O wait starts. This design is
> more complex, but it is actually "more correct".

Not sure. Same reasoning applies to comm events.

There is some good reasons not to do this. A chain of small lightweight
events quickly adds up to a heavy event.

But might make sense to drain the AsyncCall queue more than once.

> The new condition for the sawActivity loop should be:
>
> while(sawActivity && !sawHeavyEvent)

What is a heavy event in your view?

> One call is sufficient -- the async call queue drains itself completely,
> including any calls scheduled during the draining process itself, but
> the dispatchCalls() call should remain inside the sawActivity loop
> (because events create calls create "now" events create calls create
> "now" events ...).

would it be possible to make that only fire already queued calls and let
the main event loop worry about dealing with chained calls?

> Does the attached patch makes sense to you? Does it solve the "I/O
> starvation during rebuild" problem you found?

It probably solves the starvation, but I do not like that sawActivity
loop at all. It too easily creates comm starvation if one is careless
with event chains. It it is to stay then it need to be limited in number
of iterations accepted per main loop.

The patch looks good if sawActivity loop-in-the-loop is to stay. If
sawActivity loop is not to stay then this patch is not needed, but still
makes logic of the code more visible even if the result code doesn't
really mean anything particular to the main loop.

Regards
Henrik
Received on Mon Dec 03 2012 - 23:16:45 MST

This archive was generated by hypermail 2.2.0 : Tue Dec 04 2012 - 12:00:05 MST