Re: Chunked mempools, a first verdict

On Wed, May 02, 2001, Andres Kroonmaa wrote:

> You guys are confusing hell out of me. What is the API you are after?
> Public API is carved in stone:
>
> memPoolAlloc(MemPool *);
> memPoolCreate(char *, size_t);
> memPoolFree(MemPool *, void *);
> memPoolDestroy(pool);

THat makes sense.

>
> All else is bunch of hacks and needed/useful helpers.
> Eg. someone have put stats for memPool allocs/frees on the Info page.
> For that he have added two globals to the zoo:
>
> extern unsigned int mem_pool_alloc_calls;
> extern unsigned int mem_pool_free_calls;

Ok, I hate external variables, because I'm an evil "I don't want
it overwritten" person. Thats a personal thing. If it were

me I'd have a memPoolGetStats(mempool_stats_t *stats);

or something, but again, thats me.

So, no real problems there.

> Is that now part of an API?
>
> Ok, lets go though it again:
> memPoolInit() - we need that, don't we?

Yup

> memPoolConfigure - need to change mem_idle_limit

yup

> memPoolInUseCount(pool) - to get number of items in use for a pool
> memPoolCleanModule() - destroy all idle pools, at shutdown (and reconfig?)
> memPoolsTotalAllocated() - get total allocated memory, for Info page

I'd combine memPoolsTotalAllocated() with the memPoolGetCallStats().

> Now specific to chunked mempools:
> memPoolCleanIdlePools - need periodic cleanup function, with access to event.c
> to reschedule itself. Needs access to worker funcs under lib/ or must be
> implemented in full under lib/

.. or you can implemented it as a standalone function,specify in the
API that it needs to be called periodically, and then somewhere
in src/ have a wrapper to give it access to event.c .

stuff in lib/ really shouldn't know anything about src/ .

> memPoolTune - useful, to tune chunk sizes
> memPoolFlushMeters - need to get accounting meters uptodate before reports

> memPoolReport() - cachemgr reporter

Right. This dumps out the per-mempool info?
This one could be split a little - you could extend the stats
struct above to include an list of mempool info, and then
have a function in src/ which converts it to cachemgr.

> memMeterSyncHWater() - if we want high water mark accounting.
>
> Now, either funcs or data structs to get statistics reported:
> TheMeter - accounting totals for all pools
> Array Pools - to traverse perpool accounting stats for reporting

If you implement my suggestions above I don't think you need these to be
external.

> What of the above can we discard in principle??
>
> Stuff with duality due to lib/ vs. src/ split:
> memPoolInit() - is run as early as possible. To implement, you either add
> a worker func in lib/ or share Array Pools data struct and implement in src/
> Either way - 2 symbols at least
> memPoolConfigure - you either pass a new limit to a lib/ func, or share
> global variable
> memPoolCleanIdlePools() - no way you can reschedule event from under lib/
> you either have a public func to implement, or share data structs to
> implement in under src/
> memPoolCleanModule() - use of debug() forces it to be wrapped in src/ -
> 2 symbols: worker and wrapper
> memPoolReport() - needs tons of stuff from lib/, yet has to be done under src/
> as it writes to StoreEntry - 2 symbols at least.
> etc..
>
>
> I'm at total loss as to whats wrong with above public symbols.
> To undo the control function is not problem. If that is unwanted,
> I'd revert back to old style. Just please be clear on your preference.
> I just think that at least some functions are well suited for a Control
> function (like Tune, Init, Flush, CleanModule)
>

If its going in lib/ I'd say to myself "if I take the code and
any other code it requires in lib/ and use it in my program
that has nothing to do with squid's src/, would it still run?"

But, thats just me. I happen to like nicely-bounded APIs
and code reuse. :-)

Adrian

-- 
Adrian Chadd			"How could we possibly use sex to get
<adrian@creative.net.au>	  what we want?
			 	   Sex _IS_ what we want!" -- Fraser