Re: [squid-users] storeDirWriteCleanLogs() blocking queries

On tis, 2008-10-07 at 15:25 -0400, Chris Woodfield wrote:

>
> We've been noticing lately that the logrotation process is taking
> longer and longer as our caches fill up - currently, with ~18 million
> on-disk objects, we've seen it take as long as 12 seconds, during
> which time squid is not answering queries.

Yes..

> Is this still the case that the storeDirWriteCleanLogs() function is
> expected to take this long when the cache_dirs get this large?

Time is expected to grow with size yes.

> Is
> there anything that can be done to mitigate this?

Disable swap if you have any, to make sure the index data never gets
swapped out. You generally don't need swap in a Squid server.

> The issue is that we
> rotate logs fairly frequently (multiple times per hour), which
> amplifies this issue.

Indeed.

> As a workaround, we may disable the storeDirWriteCleanLogs() in
> mainRotate() and trigger it on a different signal instead. Sound like
> a reasonable workaround?

Yes.

> If so, what should the maximum time between
> rotating swap.state be?

Depends on the churn rate in your caches. The more that has changed in
the cache since swap.state was last cleaned the longer Squid will take
on an unscheduled restart.

swap.state is not used during normal operations, only as a journal of
the cache contents in case Squid gets restarted with no chance of
writing out a clean index (power failure, kernel panic, kill -9, double
seg fault, etc...)

Regards
Henrik