Re: [squid-users] Huge Cache / Memory Usage

On 16/12/10 08:17, Sunny wrote:
> Hi there,
>
> I am working on building a cache with squid 3.1.9. I've got two
> machine with 4G ram and two 500G disk each. I want to make cache as
> large as possible to maximize utilization of my two big disk.
>
> However, I soon found out I am being extremely limited by memory. lots
> of swapping starts to happen when my cache exceed 9M objects. Also

Rule #1 with Squid is: don't let it swap.
In my experience the bits that get swapped out tend to be the long term
index entries. Which are searched on every request so swapping X MB in
and out again on every request causes a major performance penalty.

> everytime I want to restart cache, it would spend a hour just to
> rescan all the entities into memory. and it just keep taking longer.

This is a sign that your index is not being saved to disk cleanly. Check
the startup process for messages indicating a "(DIRTY)" cache startup.
If present that needs to be investigated and fixed.

  The fix is usually to allow Squid sufficient time when shutting down
to disconnect from clients, helper apps and flush it's entire index into
the swap.state journal. It can then load much faster on startup instead
of re-scanning the whole disk.

> And From iostat -x -d , my two disk utilization is often below 5%
> during scan and serving, which is kind of a waste.

This is probably a good thing. Less wait time for IO, as long as the
disks don't have to spin up from idle.

>
> from some doc, I found statement that squid needs 14M (on 64 bit) for
> each 1G on disk. If that's the case, to fill 500G disk I would need
> ~8G ram just to hold the metadata.
>
> So my question is:
>
> 1. Is this statement true? Can squid somehow lookup directly on the
> disk to imporve disk utilization and reduce memory needs?

Yes it is true. As an estimated value based on ~64KB avg object size.
Your specific stuation may vary.

You just said above it took an hour to scan the disk loading all the
URLs into memory. This would need to be done on every single request.
All the index holds is the URL the name of file it came from and the
headers needed to calculate age + variant ID.

> 2. How big the cache people usually have? I think having a 500G cache
> will definitely improve hit ratio and byte hit ratio, is that true?

I just had to fix problems uncovered and annoying people with TBs of
total cache. Your 500GB is not that big nowdays, but above average.

> 3. what other optimization is needed for building huge cache?

Disable atime on the cache_dir drive(s). The cache files are easily and
often changed or fetched fresh from the network. Opinions differ about
FS journaling.

Spreading it over many disk spindles. Each cache_dir is managed
separately and a bunch can pull/push objects in parallel with less
waiting. Multiple cache_dir on a single drive *will* clash with their IO
instructions. And there is a 2^31 objects per cache_dir hard limit. So
multiple smaller disks serve better than single huge ones.

It's up to you how much you want to test and tune all this.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.9
   Beta testers wanted for 3.2.0.3