Re: [squid-users] Squid Scalability

From: Gavin McCullagh <gavin.mccullagh_at_gcd.ie>
Date: Mon, 6 Apr 2009 12:26:06 +0100

Hi,

On Tue, 07 Apr 2009, Amos Jeffries wrote:

> Gavin McCullagh wrote:

>> Mine too. The operating system is on linux software RAID1 partitions so I
>
> Ah, there we probably have the answer as to why there is so much iowait.

I'm not convinced of that. The iowait seems to grow directly as a function
of the cache size and the caches themselves are not RAIDed. You can see
that I recently reduced the cache size and got an immediate, substantial
reduction in iowait.

http://deathcab.gcd.ie/munin/gcd.ie/watcher.gcd.ie.html#Squid
http://deathcab.gcd.ie/munin/gcd.ie/watcher.gcd.ie.html#System

> You may want to find out how to determin RAID iowait vs other iowait and
> see what shows up.

That would be interesting alright. I'll see what I can find out.

> Though I only had 2x 250GB disks on 2.6GHz box RAID1, Squid maxed out
> and started seriously lagging requests under 3 users (at around ~5Mbps
> wild guess).

Did you have the cache on RAID1 or the OS or both? Hardware or software
RAID (not that my using software should improve anything of course)?

> I shifted to a *slower* 1.8GHz box with single OS-shared disk and it now
> hits serves 15 users without sweating and runs dozens of reverse-proxy
> domains as a side job.

Squid's usage of CPU time doesn't seem to be an issue for us at all so I
can well believe that.

> My review of RAID + Squid was overruled by some RAID experts with more
> experience. I'm still puzzled how they got the evidence for
> "performance: quite good" on software RAID though, maybe dual-core
> minimum, mine are both singles.

I can certainly see how putting the cache on software RAID1 is a bad plan,
but that's not what I've done and the iowait is sensitive to cache size.
I have the squid logs and the squid cache on single disk partitions. I
don't think the OS shouldn't be loading the disk too much.

Many thanks for your help on this,

Gavin
Received on Mon Apr 06 2009 - 11:26:10 MDT

This archive was generated by hypermail 2.2.0 : Mon Apr 06 2009 - 12:00:02 MDT