chunked MemPool

On 30 Mar 2001, at 12:50, Henrik Nordstrom <hno@hem.passagen.se> wrote:

> Do you mind putting this improved mempool up on SourceForge to ease
> further development and testing until merged into HEAD?

 It is now up on SF under name chunked_mempools.
 Comments and improvement suggestions very welcome.

 (note that mempools are now under lib/ making it possible to use
  mempools anywhere in the code)

 Benefits are not terribly outstanding, but still...

 HEAD:
Memory usage for squid via mallinfo():
        Total space in arena: 243952 KB
        Ordinary blocks: 238609 KB 67786 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 4368 KB 6 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 5343 KB
        Total in use: 242977 KB 100%
        Total free: 5343 KB 2%
Internal Data Structures:
        2355679 StoreEntries
 ~105 bytes / StoreEntry

 HEAD+chunked_mempools:
Memory usage for squid via mallinfo():
        Total space in arena: 137033 KB
        Ordinary blocks: 136941 KB 19 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 56584 KB 64 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 91 KB
        Total in use: 193525 KB 141%
        Total free: 91 KB 0%
Internal Data Structures:
        2350120 StoreEntries
 ~82 bytes / StoreEntry

 CPU overhead, 30sec average: (measure is in CPU-clock ticks)

 HEAD:

probe name count best average worst CPU%
---------------------------------------------------
memPoolFree 33947 213 1159 702491 0.162
memPoolAlloc 33599 195 712 364622 0.099
xcalloc 1549 381 4008 159758 0.026
xmalloc 4056 229 1333 78890 0.022
xfree 3525 117 850 113049 0.012

 HEAD+chunked_mempools:

probe name count best average worst CPU%
---------------------------------------------------
memPoolFree 25392 783 2645 7800268 0.278
 memPoolPush 25392 210 1741 7793777 0.183
memPoolAlloc 27573 724 1825 4946031 0.209
 memPoolGet 27573 131 737 4942942 0.084
xcalloc 849 309 8746 1475307 0.031
xmalloc 3746 226 1569 112201 0.024
xfree 2541 118 727 337958 0.008

memPoolClean 6665 106 194 93459 0.005
memPoolChunkNew 32 15955 268983 4925114 0.036

(probe_overhead 146468 95 328 829723 0.181)

 Chunked mempool is slower, about 2 times.
 Compiled without any optimisations. Still very comparable.
 Interestng to note that memPoolAlloc is memPoolGet +stats
 gathering. So most of the time goes to incrementing counters,
 probably very well optimised by compiler.

 I'm just wondering if it makes any sense to use some hash
 functions for finding chunks?

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Delfi Online
 Tel: 6501 731, Fax: 6501 708
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Tue Apr 10 2001 - 03:03:06 MDT