Bigger Squid Memory Usage ?

From: Karsten Gessner <karsten@dont-contact.us>
Date: Mon, 17 Jul 2000 16:58:02 +0200

In FAQ:
8.1 Why does Squid use so much memory!?
Squid uses a lot of memory for performance reasons. It takes much, much longer to read something from disk than it does to read directly from memory.

A small amount of metadata for each cached object is kept in memory. This is the StoreEntry data structure. For Squid-2 this is 56-bytes on "small" pointer architectures (Intel, Sparc, MIPS, etc) and 88-bytes on "large" pointer architectures (Alpha). In addition, There is a 16-byte cache key (MD5 checksum) associated with each StoreEntry. This means there are 72 or 104 bytes of metadata in memory for every object in your cache. A cache with 1,000,000 objects therefore requires 72 MB of memory for metadata only. In practice it requires much more than that.

Does this always mean that my 2.368.872 StoreEntries must end up in 226.950 KB

Connection information for squid:
        Number of clients accessing cache: 338
        Number of HTTP requests received: 82353
        Number of ICP messages received: 0
        Number of ICP messages sent: 0
        Number of queued ICP replies: 0
        Request failure ratio: 0.00%
        HTTP requests per minute: 98.5
        ICP messages per minute: 0.0
        Select loop called: 936471 times, 53.579 ms avg
Cache information for squid:
        Request Hit Ratios: 5min: 53.7%, 60min: 51.9%
        Byte Hit Ratios: 5min: 42.4%, 60min: 32.0%
        Storage Swap size: 20778437 KB
        Storage Mem size: 16340 KB
        Storage LRU Expiration Age: 90.00 days
        Mean Object Size: 8.77 KB
        Requests given to unlinkd: 2635
Median Service Times (seconds) 5 min 60 min:
        HTTP Requests (All): 0.14252 0.10857
        Cache Misses: 0.37825 0.24524
        Cache Hits: 0.05633 0.05331
        Near Hits: 0.10857 0.24524
        Not-Modified Replies: 0.04277 0.04047
        DNS Lookups: 0.01609 0.02231
        ICP Queries: 0.00000 0.00000
Resource usage for squid:
        UP Time: 50174.876 seconds
        CPU Time: 482.400 seconds
        CPU Usage: 0.96%
        CPU Usage, 5 minute avg: 0.98%
        CPU Usage, 60 minute avg: 1.57%
        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 194077
Memory usage for squid via mallinfo():
        Total space in arena: 227448 KB
        Ordinary blocks: 226950 KB 932 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 2544 KB 4 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 498 KB
        Total in use: 229494 KB 101%
        Total free: 498 KB 0%
Memory accounted for:
        Total accounted: 186284 KB
File descriptor usage for squid:
        Maximum number of file descriptors: 1024
        Largest file desc currently in use: 72
        Number of file desc currently in use: 60
        Files queued for open: 0
        Available number of file descriptors: 964
        Reserved number of file descriptors: 100
        Store Disk files open: 0
Internal Data Structures:
        2.368.872 StoreEntries
          2953 StoreEntries with MemObjects
          2944 Hot Object Cache Items
        2368631 Filemap bits set
        2368631 on-disk objects
Received on Mon Jul 17 2000 - 09:03:10 MDT

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:54:32 MST