Re: Seeking enterprise ideas for Squid

On Tue, Oct 08, 2002, GV wrote:

> Network I/O: Instead of polling client connections,
> Squid could employ an interrupt mechanism to determine
> the next “ready” client. We implemented RT Signals
> (targeted at improving performance by reducing network
> I/O), and got a lot of help and support from the Open
> Source developers. (The intent here is not to glorify
> RT Signals vis-à-vis polling. Rather, it is an example
> of the kind of ideas we are looking for in order to
> bring Squid performance progressively closer to the
> “desired enterprise level performance” as defined by
> some of our potential clients).

The commloops branch is focused on refactoring and implementing
a replacement IO mechanism for squid - once all of the
direct reads are replaced with a callback IO mechanism
then many, many other network IO mechanisms can be
introduced. Yes, this possibly includes efficient
support for win32 completion port io.

> Disk I/O: We have not looked at this issue in great
> detail. There does seem to be an opportunity to
> improve I/O performance by using raw I/O instead of
> going through the file system. There may be other
> alternatives as well.
>

Squid's storage/indexing layer needs a little more work
before its able to support a higher disk IO throughput.

> Multithreaded Squid: This comes up during discussion
> with enterprise customers. We are not sure it is
> necessarily the only way to go. But the economics of
> running Squid on a single box (duly partitioned)
> vis-à-vis running multiple instances on multiple
> systems can get pretty overpowering. Does the
> development community have any thoughts to share on
> the potential of a multithreaded Squid product? Or are
> there ways to run multiple instances of Squid on a
> properly partitioned SMP system, with acceptable
> levels of scaling in thruput and response time?

Threading CAN work, but not one thread per connection.
That doesnt' scale.

> We are not looking for complete agreement on any of
> the ideas stated above. However, because there will be
> different views, what do people think of the notion of
> an enterprise edition of Squid, and a standard
> edition? Or is the overhead of two Squid versions
> (multiplied by the number of platforms) simply not
> worthwhile?

Well, I'd like squid to be suitable for both enterprise
and home users. Its certainly possible.

> On a related note, is it appropriate to discuss
> proprietary vendor features on this discussion list?

Well, it depends how proprietary you'd like it to be. :)

A blantant plug here - Robert and I have been working quite
hard on improving the comm layer for squid (especially
Robert's refactoring work - very very helpful!) which will
provide quite significant improvements in Squid's network
performance. There are other areas (I'll be working
on the store layer next, along with implementing an efficient
and stable COSS) but our main issue is lack of time.
We all have to eat, and my current employers use squid but
not enough to have me working on it 12 hours a day.

Adrian
Received on Wed Oct 09 2002 - 01:33:07 MDT