Re: bucket randomisation for storeMaintainSwapSpace()

mark@aone.com.au writes:

>Back around 1.1.6 a change was introduced to randomise the order
>of processing for storeMaintainSwapSpace(). I assume that the
>reason for adding this randomisation was to overcome the problem
>whereby an unstable squid would never scan the entire cache. (say squid
>core dumped every 12 hours, the last half of the buckets would not get
>scanned). I would think randomising the starting point for the scan
>would have been sufficient, randomising the entire sequence seems
>reasonable too.
>
>I don't understand the reason for rerandomising the sequence after a
>full scan. qsort()ing 65k random numbers is relatively time consuming
>on any box (on an ultra 170 .75 seconds. On an alpha >.25 seconds), and
>you don't really want that sort of delay whilst in production mode if
>not necessary.
>
>I'd suggest the following change be applied, but if there's a reason
>for the re randomisation of the processing sequence, I'd be interested
>to hear.

The rationale is that the cache load follows the 24 hour
busy/not-so-busy cycle. We've calculated the scan rate so that the
entire cache is scanned roughly every 24 hours. This means that you'll
always be scanning the same buckets during the busy period (assuming
your cache stays running longer than 24h). It may drift a little, but
probably not enough.

If you always scan the same buckets during the busy period then those
buckets will be mostly empty and the buckets scanned during the idle
period will be overly full.

Duane W.
Received on Tue Jul 29 2003 - 13:15:40 MDT