Re: compute swap_file_sz before packing it

On 10/27/2009 02:41 PM, Henrik Nordstrom wrote:
> The object size field in STORE_META_STD should be ignored. Got broken
> many years ago (1997 or so), and should be recalculated by using
> STORE_META_OBJSIZE or alternatively the on-disk object size.

Rock code can ignore the swap_file_sz field stored in STORE_META_STD but
the core code is still using StoreEntry::swap_file_sz so I have to set
that field to the right value when building an index.

To compute StoreEntry::swap_file_sz, I will add up the ported
STORE_META_OBJSIZE value and the swap_hdr_len set by StoreMetaUnpacker.
Would you compute it differently?

What should I do if STORE_META_OBJSIZE is not known? Does this question
itself imply that each store that wants to rebuild an index has to store
the final object size somewhere or update the STORE_META_OBJSIZE value?

>> Moreover, STORE_META_OBJSIZE has the following comment attached to its
>> declaration: "not implemented, squid26 compatibility" and appears to be
>> unused...
>
> Right.. should be forward ported. [attached]

Thank you for porting this.

>> Neither approach works for Rock store because Rock store does not have a
>> swap state file like COSS and does not use individual files like UFS.
>> That is why it has to rely on the "file" size information supplied by
>> the core. Perhaps there is a better way of getting that information, but
>> I do not know it.
>
> STORE_META_OBJSIZE is the object size (if known) not including TLV
> headers, and is generally what you need to know in order to access the
> object.

What happens to STORE_META_OBJSIZE if the object size is not yet known
at the time when Squid start swapping content to disk?

> Long term objects should be split in TLV + HTTP Headers (probably part
> of TLV) + Content, but that's another topic..

> Actual file storage size is more a business of the cache_dir than the
> core..

I agree but the current core code does not :-).

>>> + // so that storeSwapMetaBuild/Pack can pack corrent swap_file_sz
>>> + swap_file_sz = objectLen() + mem_obj->swap_hdr_sz;
>>> +
>
> objectLen() MAY be -1 here...

The whole patch is not needed if we start relying on STORE_META_OBJSIZE,
I guess.

Thank you,

Alex.
Received on Tue Oct 27 2009 - 21:51:02 MDT