Re: Storing partial responses

From: Henrik Nordstrom <hno@dont-contact.us>
Date: Thu, 14 Dec 2000 04:53:31 +0100

Joe Cooper wrote:

> Requires a disk format or swap.state change, probably. (Whether we
> break it up or put the pieces all in one place.)

Well. luckily the disk format is extensible. We can quite easy
add/change the meta format of new objects without breaking compability
with old ones. And there is also one or two flag bits left if we need to
have anything added to the in-core index without needing to change
swap.state.

> Breaks the storetree stuff that is going to be merged in from the
> squidng project. storetree does a hash of the URL for storage
> indexing...and keeps no record of what was just stored.

Well.. I am not sure it needs to break storetree. The format change can
be done within the object files, still indexed on the URL on a higer
level as before. The filesystem does not need to know that the object
contains ranges.

> If the object
> is not complete, reiser_raw won't know that and will serve it
> incomplete. So a new field would also be needed in reiser_raw. Which I
> guess applies to the standard object store as well...so chalk that one
> up to "necessary change" for version DEVEL2.5.

Squid will know from the stored metadata in the file.

> The idea of rewriting files and expiring the 'pieces' as new pieces come
> in, until we have a complete object. Adds significant write overhead,
> but keeps the index simple and objects end up being as large as they can
> be...reducing read and seek overhead.

Hmm.. you still needs to keep an index of which parts you have stored so
you know which ranges might be there..

/Henrik
Received on Wed Dec 13 2000 - 22:07:07 MST

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:13:04 MST