Cache is exceeding the specified limit

From: Steve Snyder <swsnyder@dont-contact.us>
Date: Sat, 22 Jul 2000 09:57:05 -0500

My Squid (2.3-S3/S4) has grown to exceed the disk space I specified as
the limit to use on my (Linux v2.2.16) system. Why?

Below you'll see that I've specified 256MB of disk space for the
cache and 98% as the swap_high point. That being the case, my cache
shouldn't get much beyond (256901KB = 0.98 * 262144KB), right? Yet
here I am with this:

  # du -s /var/spool/squid/00
  264620 /var/spool/squid/00

I am aware of the LRU no-deletion-of objects bug in 2.3S3, but that
shouldn't be a factor in my problem because not only did I apply the
patch, but I'm using the LFUDA algorithm.
 
A few things I should note:

  1. The RAM-related items show here are low because I just replace
    2.3S3 with 2.3S4 last night.
  2. Ditto the number of requests given to unlinkd.
  3. I've specified 8KB as the average object size, but the actual
    size is 9KB.

First, a few (relevant?) selections from my squid.conf:

  cache_dir ufs /var/spool/squid 256 1 254
  cache_swap_low 95
  cache_swap_high 98
  store_avg_object_size 8 KB

Next, my startup messages from cache.log:
 
  Starting Squid Cache version 2.3.STABLE4 for i686-pc-linux-gnu...
  Process ID 21713
  With 1024 file descriptors available
  DNS Socket created on FD 4
  Adding nameserver 127.0.0.1 from /etc/resolv.conf
  Adding nameserver 24.4.162.33 from /etc/resolv.conf
  Adding nameserver 24.4.162.34 from /etc/resolv.conf
  helperOpenServers: Starting 4 'squid_redirect' processes
  Unlinkd pipe opened on FD 13
  Swap maxSize 262144 KB, estimated 32768 objects
  Target number of buckets: 655
  Using 8192 Store buckets
  Max Mem size: 24576 KB
  Max Swap size: 262144 KB
  Store logging disabled
  Using LFUDA disk replacement policy
  Rebuilding storage in /var/spool/squid (CLEAN)
  Set Current Directory to /var/spool/squid
  Loaded Icons.
  Accepting HTTP connections at 0.0.0.0, port 3128, FD 14.
  WCCP Disabled.
  Ready to serve requests.
  Done reading /var/spool/squid swaplog (23628 entries)
  Finished rebuilding storage from disk.
      23628 Entries scanned
          0 Invalid entries.
          0 With invalid flags.
      23628 Objects loaded.
          0 Objects expired.
          0 Objects cancelled.
          0 Duplicate URLs purged.
          0 Swapfile clashes avoided.
    Took 0.8 seconds (28000.2 objects/sec).
  Beginning Validation Procedure
    Completed Validation Procedure
    Validated 23628 Entries
    store_swap_size = 212768k
  storeLateRelease: released 0 objects
  
Finally, the output of "client mgr:info":
 
  Squid Object Cache: Version 2.3.STABLE4
  Start Time: Sat, 22 Jul 2000 05:27:14 GMT
  Current Time: Sat, 22 Jul 2000 14:24:22 GMT
  Connection information for squid:
          Number of clients accessing cache: 3
          Number of HTTP requests received: 1134
          Number of ICP messages received: 0
          Number of ICP messages sent: 0
          Number of queued ICP replies: 0
          Request failure ratio: 0.00%
          HTTP requests per minute: 2.1
          ICP messages per minute: 0.0
          Select loop called: 71261 times, 452.258 ms avg
  Cache information for squid:
          Request Hit Ratios: 5min: 0.0%, 60min: 68.8%
          Byte Hit Ratios: 5min: 100.0%, 60min: 9.9%
          Storage Swap size: 213594 KB
          Storage Mem size: 524 KB
          Storage Replacement Threshold: 0.000000
          Mean Object Size: 9.00 KB
          Requests given to unlinkd: 23
  Median Service Times (seconds) 5 min 60 min:
          HTTP Requests (All): 0.00091 0.03622
          Cache Misses: 0.00091 0.72387
          Cache Hits: 0.00000 0.00815
          Near Hits: 0.00000 0.18699
          Not-Modified Replies: 0.00000 0.01955
          DNS Lookups: 0.00000 0.00094
          ICP Queries: 0.00000 0.00000
  Resource usage for squid:
          UP Time: 32228.381 seconds
          CPU Time: 3.700 seconds
          CPU Usage: 0.01%
          CPU Usage, 5 minute avg: 0.00%
          CPU Usage, 60 minute avg: 0.01%
          Maximum Resident Size: 0 KB
          Page faults with physical i/o: 1553
  Memory usage for squid via mallinfo():
          Total space in arena: 5489 KB
          Ordinary blocks: 5371 KB 112 blks
          Small blocks: 0 KB 0 blks
          Holding blocks: 176 KB 1 blks
          Free Small blocks: 0 KB
          Free Ordinary blocks: 118 KB
          Total in use: 5547 KB 101%
          Total free: 118 KB 2%
  Memory accounted for:
          Total accounted: 2230 KB
  File descriptor usage for squid:
          Maximum number of file descriptors: 1024
          Largest file desc currently in use: 14
          Number of file desc currently in use: 15
          Files queued for open: 0
          Available number of file descriptors: 1009
          Reserved number of file descriptors: 100
          Store Disk files open: 0
  Internal Data Structures:
           23748 StoreEntries
             108 StoreEntries with MemObjects
             107 Hot Object Cache Items
           23738 Filemap bits set
           23738 on-disk objects

Thank you.
Received on Sat Jul 22 2000 - 09:00:12 MDT

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:54:34 MST