Re: Chunked mempools, a second verdict

On 3 May 2001, at 17:30, Alex Rousskov <rousskov@measurement-factory.com> wrote:

> On Fri, 4 May 2001, Andres Kroonmaa wrote:
>
> > As it is currently, it is somewhat clumsy to determine NextPool
> > based on *prev_pool. Would require linear search.
...
> There are probably other ways to avoid linear search.

 yep, nonissue.

> > > or even one using a abstract iterator type
> > >
> > > extern MemPoolIterator * memPoolIterate(void);
> > > extern MemPool * memPoolGetNextPool(MemPoolIterator **);
> > > and optionally (in which case one * above goes away)
> > > extern void memPoolIterateDone(MemPoolIterator **)
> >
> > Why would that be good? (I've not seen something like that yet)
> > Sounds complex to me. At least not intuitive.
>
> It is a standard OO iterator, converted to C. I agree that it looks
> more complex in C. The real advantage of those things is that you can
> pass an iterator to a third function (algorithm), and that function
> would iterate "anything" doing "something" on it. You will not see
> many of such things in Squid yet, but it is a standard STL technique.
> As you know, I stopped short of proposing this; I was afraid to be
> pummeled for forcing OO/C++ ideas onto Squid code.

 How do you use such thing?
 MemPoolIterator *Iter = memPoolIterate(void);
 while (pool = memPoolGetNextPool(&Iter)) {
         ...
 }
 What is the returned MemPoolIterator ? MemPool * ? xmalloc()ed data?
 IMO its only confusing. I'd prefer simple First/Next approach.
 Would we need such abstraction in memPools? All we want is a way to
 pass over all pools.

> > > > extern void memPoolCheckRates(); /* stats history checkpoints */
> >
> > see my other post about perpool alloc rates/sec column in cachemgr.
> > currently, memReport is modifying memMeter structs it receives from
> > memPoolGetStats(), by copying current counter aside. This call is
> > meant to ask the library to do that internally for us.
> > To maintain historical stats outside the pool itself would be
> > trouble, and no easy way to get notified when pool is destroyed.
>
> Still not sure why do you need this call. Why can't the library do it
> on its own, when needed (either via memPoolUpdateTime that Henrik
> banned or from inside of other public functions).

 like what functions? Alloc/Free out of question, they are tuned for
 performance, and any overhead is unwanted there. GetStats? Iterate?
 We don't have much funcs left in fact. If we manage to make pools
 selfcleaning, we'd loose currently best place: memPoolClean.

 What if not a single memPool func is called for several hundred rate
 measurement intervals in a row, after say spike of 1M allocs? What
 meaning would be left to the rate counters?

 Basically only for a determined sampling interval. In this sense,
 memPoolUpdateTime would be loveliest, sort of heartbeat provided
 by the app. All timerelated stuff could be done there, including
 memPoolClean.
 Checkpointing api is nice because it is clear what it is, and is
 unrelated to other stuff.

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Delfi Online
 Tel: 6501 731, Fax: 6501 708
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Fri May 04 2001 - 02:17:33 MDT