Re: storeLowestMemReaderOffset

On Mon, 2002-10-21 at 00:41, Henrik Nordstrom wrote:
> I don't.

My WAG is that it was associated with disk clients getting hot stored
once in memory, or something like that.

Anyway. that is now a 2 line function, so it doesn't really matter :].

On a related note though: I'd like to propose a change to the memory
caching logic:

Currently:
we have n store clients that read data, and a swapout logic routine that
'chases' the bottom most store client, and finally we free the swapped
out fragments once swapped out if the object is not a hot object.

Proposed:
The swapout logic becomes a store client in it's own right.
Each fragment is marked as discardable once swapped out (if not a hot
object).
If a fragment is needed to satisfy a store client, it is retrieved from
disk to memory, and immediately marked as discardable.
Discardable fragments are discarded when memory is needed, according to
a policy (say lru to start with)

The change will involve a number of steps, these can be planned later.
The important aspects are:
1) Restarting the cache will not prevent objects from being 'hot
objects' - that is objects retrieved from disk can be cached in memory.
2) Objects with multiple readers that have one very slow and one fast
will not consume inordinate amounts of memory - the intermediate pages
will be able to be swapped to disk and back (if appropriate).
3) It is a critical step towards range combining, where we need to merge
upstream and disk based data, by allowing disk data to be brought into a
normal MemObject structure.
4) The current special logic for swapping out objects can go away and be
replaced by simply starting a swapout client off. This makes the swapout
process more loosely coupled, which will be important if/when we start
updating disk metadata / caching ranges.

Any objects to this design change? If not I'll implement it in
fix_ranges and bring it across to HEAD from there.

BTW: The fix_ranges effort is coming along nicely, we are nearly at the
point of supporting range requests without always returning the full
object again.

Rob