Re: Squid-smp : Please discuss

>>> I think we should take this on-list so the others with more detailed
>>> knowledge can give advice in case I have my facts wrong about
>>> AsyncCalls...
>>
>> I am afraid this discussion focuses on a small part of a much bigger
>> problem so finalizing the design decisions here may be counter
>> productive until we have an agreement on how to split Squid into
>> threads in general.
>
> Aye. Thats the idea.
>
>>
>> There are a few different ways to partition Squid and most of them
>> have been discussed in the past. I am not sure whether the discussions
>> have ever reached a consensus point. I am also not sure there is
>> consensus whether we should design for 2 cores, 8, cores, 16 cores, 32
>> cores, or more, or all of the above?
>
> The partitioning discussion must have happened well before my time. The
> last few years its been consensus that the components get partitioned into
> SourceLayout components and AsyncCalls codepaths.
> Further partitioning we have not discussed recently.

I'm going to kick-start a new round then. If the approach has already
been discussed, please forgive me and ignore this post.
The idea is.. but what if we tried using a shared-nothing approach?

Quick run-down: there is farm of processes, each with its own
cache_mem and cache_dir(s). When a process receives a request, it
parses it, hashes it somehow (CARP or a variation thereof) and defines
if should handle it or if some other process should handle it. If it's
some other process, it uses a Unix socket and some simple
serialization protocol to pass around the parsed request and the file
descriptor, so that the receiving process can pick up and continue
servicing the request.

There are some hairy bits (management, accounting, reconfiguration..)
and some less hairy bits (hashing algorithm to use, whether there is a
"master" process and a workers farm, or whether workers compete on
accept()ing), but on a first sight it would seem a simpler approach
than the extensive threading and locking we're talking about, AND it's
completely orthogonal to it (so it could be viewed as a medium-term
solution while AsyncCalls-ification remains active as a long-term
refactoring activity, which will eventually lead to a true MT-squid)

Please forgive me if it's a 5AM sleep-deprivation-induced brain-crap,
or if this approach was already discussed and discarded.

-- 
    /kinkie