Re: subscribe squid-dev interest.

From: Robert Collins <robertc@dont-contact.us>
Date: 19 Nov 2002 22:50:17 +1100

On Mon, 2002-11-18 at 19:40, l. walsh wrote:
> Hi, I'm Linda. I've been known to do development. I've been known to
> have
> opinions (informed, intuitive or My birth sign is
> in Leo and rising in Sag. How's that for an intro? Ok, that wasn't
> the intro whoever sits in judgement was looking for?

Good enough, but I don't sit in judgement, you'll have to wait for that
to occur...

> I'm interested in performance, extensibility and ease of use.

Neato.

> With squid, I'm wondering if anyone has done broken down where delays
> are. Like when a user request comes in, where does the time go before
> the user gets back an answer. How much is spent on waiting for DNS
> lookups, how much time before a remote system accepts the connection,
> how much time did the request take to send (likely close to zero in most
> cases), how much time was spent waiting for the user request to be
> returned and how much time to feed it back to the user? How much time
> is spent writing log files? For example with 'diskd', cache writes
> are moved into another process because disk writes are deemed to take
> too long, but are log files written synchronously? I assume reads from
> hosts and writes to clients are non-blocking.

There have been many analyses done. There are some generally known areas
for improvement, and many areas where squid is simply inefficient
internally, and could *really* use some maintenance. Also, it's
flexability allows folk to misconfigure it (see for instance the recent
emails regarding ACL types on squid-users.)

> I'd like to see such performance stats to be builtin, user configurable
> since many of the answers are going to vary depending on the the
> particular
> workload and hardware. But inclusion of such information allows a user
> to find hot spots and tune the server for their particular load.

This would be very cool. Some aspects of this already exist, but a clear
cachemgr page would be most excellent. I presume you are thinking of
something like:

Mean elapsed time to:
permit or deny request: 0.02 ms
redirect request : 0.1 ms
establish a connection to a server: 45.03 ms
....

?

> I'm also more immediately in having integrated pre and post filter
> plugins.
> Redirect provides essentially user output filtering before it gets to
> the squid process, but to my knowledge, nothing provides prefiltering
> on the content coming back from the host (or net) before it goes into
> the squid cache and there also doesn't seem to be a way to plug in a
> post filter that filters content coming out of squid but before it is
> returned to the client. Forgive me if these plugins are already
> provided
> but when I mentioned the concept on the user list, I got a bunch of
> blank
> looks.

Check out the iCAP patches for squid, they are a 'standard' to do that.

Also there are various projects to do specific things in squid - and
some generic projects like the clientStreams code in 3.0 that enable
such alterations to be more structured.

> I'd also like to see easier keyword implementation to log all traffic
> to/from specific clients in a 'recorder' type fashion. I have some
> 'smart devices' that want to talk to the web -- including some programs
> on my winPC's that want to talk to various sites on a regular basis or
> they
> get 'cranky'. I don't want to prevent the communication, but I do want
> to
> monitor what is going on. In the case of the devices, it would help
> with
> writing compatible home servers, for example.

Another good idea. There are several log alteration projects around, we
are looking for a sponsor to make the most generic one 'happen', which
is very similar to what you suggest (in fact, it's a superset in all but
one aspect - the recording of all data).

> I'm not sure how inter-client communication is currently done, but
> in some cases, a client library that uses shared memory might be more
> efficient (or then again may not be...heck if I really know anything
> until
> I've tried it).

Yep. We use that with diskd. I'm currently rewriting the IPC support
routines to be more modular, but that shouldn't interact badly with what
you mention.

Rob

Received on Thu Nov 21 2002 - 11:35:43 MST

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:18:48 MST