Re: [squid-users] COSS improvements

Hi everyone,

my exeprience with these patches:

At first:

COSS seems to be stable, it doesn't crash so often any longer.

I had the following cache_dir lines:

cache_dir coss /dev/sdb1 32760 block-size=2048 max-size=32768
cache_dir aufs /var/squid/cache1 38000 256 256 max-size=16777216
cache_dir aufs /var/squid/bigcache 25000 16 16

The 5min average load is up to 220 requests per second at a cpu load average of about 24 percent.

I get many "Failed to unpack meta data errors" (about one every 5 seconds)
I have used the cache before with squid-2.6.STABLE2 without Steven Wilton's patches. It wasn't complete filled up before.

Now, at stripe 29000 (of 32760), cachemgr storedir says:

-----------------------------------------
Store Directory Statistics:
Store Entries : 6085603
Maximum Swap Size : 98058240 KB
Current Store Swap Size: 90869390 KB
Current Capacity : 93% used, 7% free

Store Directory #0 (coss): /dev/sdb1
FS Block Size 2048 Bytes

Maximum Size: 33546240 KB
Current Size: 33546242 KB
Percent Used: 100.00%
Current load metric: 0 / 1000
Number of object collisions: 0
Pending operations: 0 out of 128
Flags:
Pending Relocations: 0
Stripe: 0, lockcount: 0, numobjects 209, flags: NOTFULL,NOTWRITING,NOTWRITTEN,MEMONLY
Stripe: 1, lockcount: 2, numobjects 237, flags: FULL,NOTWRITING,NOTWRITTEN,MEMONLY
Stripe: 29618, lockcount: 0, numobjects 137, flags: NOTFULL,NOTWRITING,NOTWRITTEN,DISK
Pending Relocations:

Store Directory #1 (aufs): /var/squid/cache1
FS Block Size 4096 Bytes
First level subdirectories: 256
Second level subdirectories: 256
Maximum Size: 38912000 KB
Current Size: 35020964 KB
Percent Used: 90.00%
Current load metric: 100 / 1000
Filemap bits in use: 796921 of 4194304 (19%)
Filesystem Space in use: 34515696/43237588 KB (80%)
Filesystem Inodes in use: 0/0 (0%)
Flags: SELECTED
Removal policy: lru
LRU reference age: 7.78 days

Store Directory #2 (aufs): /var/squid/bigcache
FS Block Size 4096 Bytes
First level subdirectories: 16
Second level subdirectories: 16
Maximum Size: 25600000 KB
Current Size: 22302184 KB
Percent Used: 87.12%
Current load metric: 100 / 1000
Filemap bits in use: 370 of 16384 (2%)
Filesystem Space in use: 25556692/29727344 KB (86%)
Filesystem Inodes in use: 0/0 (0%)
Flags:
Removal policy: lru
LRU reference age: 21.48 days
---------------------------------------------------

The cachemgr storedir interestingly says, that the cache is 100 percent used, although the stripes greater than 29618 should be completely empty.

The number of files in Store Directory #1 (aufs) - /var/squid/cache1 is increasing. Before reaching the "100% full" in cachemgr storedir were about 500,000 entries in it.

Only a small part of the requests are stored in the COSS dir since "reaching 100 percent full".

I have also written this as a bug report as bug #1734.

Greetings,

Martin Stolle

-----Ursprüngliche Nachricht-----
Von: Adrian Chadd [mailto:adrian@creative.net.au]
Gesendet: Donnerstag, 3. August 2006 13:30
An: squid-users@squid-cache.org
Betreff: [squid-users] COSS improvements

Hi everyone,

Steven Wilton has contributed a whole swath of COSS fixes which
provide even more dramatic performance fixes and some stability
fixes to the codebase.

Those who are trialing out COSS in Squid-2.6 should grab tomorrow's
snapshot and give it a whirl. I suggest you read and re-read the
COSS comments in the squid configuration file to get an idea
of the newly available "knobs".

Adrian
Received on Tue Aug 15 2006 - 23:55:18 MDT