LogHelpers

Hello Squid Dev List,

I've been modifying squid-2.5.STABLE10 to allow log data to be passed to a
external helper program in much the same way as an external acl or
redirector. This gives me a lot more freedom in how I log proxy access.

Basically, using the redirector and helper code as a guide, I've written
functions that create a logHelper object (well, struct) and using ipcCreate()
it forks a child and passes data using the normal comm_write functions in the
same way as the normal helpers.

Then, instead of calling logfilePrintf, you call a logHelper function that
passes the data to the loghelper process.

This works, for the most part. I am running into a issue, and I am not able to
guess the best way to fix it. Also, I am hoping someone with more knowledge
of the squid code can cast a critical eye on the attached patch, and make
some suggestions.

The issue I am having, is that comm_write occasionally throws the following
debug message into the cache.log:
fd_table[31].rwstate != NULL
At the same time, a line fails to be passed to my loghelper program. After
some simple load testing it became apparent that this only happened when
squid is under a reasonably high load. Would I be right in thinking that the
error occurs when one line is still being written and another line appears?

I've tried to remove calls to commSetNonBlocking() when the wfd and rfd for
the loghelper is setup, in the hopes that non-blocking IO would mean that
squid waits until one line is handled before sending another (I assume this
is what happens when with the normal access log).

The redirector and acl helpers get around this by using multiple helpers and a
request queue (this is a bit of a guess), but I wish to have only one
loghelper if I can help it. Is there a way I can set up my loghelper to
prevent this from occurring?

Thanks for any help you can give,

Matthew Smith