RE: [squid-users] PLEASE HELP: Large cache deployment and squid

In case this gives some idea. As soon as i started the L4 switching to point
to squid server, the system usage, i.e. squid usage jumped to and hit 99%.
This was about 12 requests/second.

   procs memory swap io system
cpu
 r b w swpd free buff cache si so bi bo in cs us sy
id
 0 0 0 0 118180 51944 685940 0 0 0 48 119 135 0 5
95
 0 0 0 0 118176 51944 685940 0 0 0 0 101 145 0 0
100
 1 0 0 0 118176 51944 685940 0 0 0 2 111 134 0 3
97
----------- requests startes going in
 2 0 0 0 118044 51944 685960 0 0 4 0 247 38 0 82
18
 1 0 0 0 117848 51944 686044 0 0 5 18 221 36 0 77
23
 2 0 0 0 117644 51944 686132 0 0 16 45 241 38 0 80
20
 2 0 1 0 117628 51944 686140 0 0 0 6 228 3 0 99
1
 2 0 1 0 117284 51944 686244 0 0 1 22 219 43 0 80
20
----------- requests stopped to keep the server alive
 2 0 0 0 117256 51944 686264 0 0 0 58 193 34 0 84
16
 0 0 0 0 117244 51944 686264 0 0 0 6 119 126 0 11
89
 1 0 0 0 117228 51944 686272 0 0 0 26 115 139 0 5
95
 1 0 0 0 117216 51944 686280 0 0 0 18 116 142 0 4
96
 0 0 0 0 117188 51944 686292 0 0 0 26 117 135 0 5
95
 1 0 0 0 117172 51944 686292 0 0 0 13 124 140 0 5
94
 1 0 0 0 117168 51944 686296 0 0 0 10 125 131 0 7
93
 0 0 0 0 117168 51944 686296 0 0 0 1 111 135 0 2
98

May be the server board chipset is not supported, and that high IO is giving
a problem. When the memory cache is low, then no problems, it works like a
breeze.

Mitesh

-----Original Message-----
From: Mitesh P Choksi [mailto:mitesh.choksi@stcl.com]
Sent: 25 July 2001 12:31
To: 'Chemolli Francesco (USI)'; squid-users@squid-cache.org
Subject: RE: [squid-users] PLEASE HELP: Large cache deployment and squid

Hi Chemolli,

Thanks for the direction. I tried with --enable-time-hack and
with --disable-time-hack. but did not find any difference. The only thing I
see here is that as soon as the pagecache increases to 400+MB even a single
request can drive the squid CPU utilization to 5% or more.

Can you please send me the command line that you have used on teh system. I
have also read something about SMB and non-RAM limiting pagecache giving
some problem. Maybe that it's the kernel that will require tuning rather
than squid.

My question is how do I find out what is the problem. Is there anything in
squid that can let me know this.

Moreover, does gettimeofday() and poll() fire even if I have /dev/null in
the cache logs. If so than i can spare some CPU time by disabling logs.

Thanks for any input in advance.

Mitesh

-----Original Message-----
From: Chemolli Francesco (USI) [mailto:ChemolliF@GruppoCredit.it]
Sent: 25 July 2001 10:03
To: 'mitesh.choksi@stcl.com'; squid-users@squid-cache.org
Subject: RE: [squid-users] PLEASE HELP: Large cache deployment and squid

> PLEASE HELP
>
> I have been trying to post a questoin about very large cache
> to the list but
> have not succeeded.
>
> Anyways, I have the following situation, and don't know if
> anybody has faced
> it
> as well.
>
> 2 * 1GHz PIII
> Intel Server Motherboard (ServerWorks PCI compiled in the kernel.)
> 1GB RAM
> cache_mem = 2MB

How come this little cache_mem? You have plenty of RAM, use it...

> pagecache = 2 15 75
> SCSI HDD
> cache_dir = 3 * 13GB on different disks
> linux kernel = 2.4.2-2 smp (recompiled to remove all
> unnecessary stuff and )
> squid version = 2.4STABLE1 (from redhat rawhide) (had to
> install openssl as
> well)
>
> After 12 hours of use even 150+ request / minute (yes
> minutes) squid takes
> up 70%+ CPU.

I have half your CPU speed and manage 150+ req/sec (280 during peak-time).
I don't get those kinds of loads.

> It has also happened to me with 2.3STABLE4 (from default
> rh7.1 linux cd)
> kreiserfsd taking up 10% cpu at times

Check out Joe Cooper's performance tuneups (they are referenced
from somewhere on the Squid site).
The only case I had of such a CPU usage was when using a broken
--enable-time-hack.
I'd suggest that you rebuild squid from scratch on your system (from the
tarball)

> memory cache goes to 555MB+ that's what vmstat shows. squid
> takes up ~20MB
> as shown by top

the 500Mb part is not strange, since Linux allocates all unused RAM to that.

> Strace showed me the following after the 12 hour check (poll %time is
> surprising)
> 100.11 -2088636.745941 -440 2907882 poll

Squid is spinning like crazy in the select loop.
Might be an effect of --enable-time-hack.

> All documents I read says Squid does not require more
> processing power, yet
> I gave it 2 x 1GHz. Is that a problem?

That's much more than you'll need unless you do more than
400 req/s.

--
	/kinkie