Re: Meta-data storage

From: Yee Man Chan <ymc@dont-contact.us>
Date: Sun, 21 Apr 2002 06:26:42 -0700 (PDT)

Hi Henrik,

   I think ETag is more like an identifier for a
certain version of the content located at a particular
URL. Do you still use URL (or more accurately, MD5 of
method+URI) to lookup a StoreEntry?

   The idea I am pursuing is like you compute the MD5
of each incoming transfer, and save it to the disk
based on the MD5. Since there are mirroring activity
going on somewhere, it is possible that content from
different URLs yield the same Content MD5. In that
case, I want to only store one copy of the content and
save all the mappings from those URLs to this MD5.
Therefore I need the metadata to keep track of these
mappings.

Regards,
Yee Man

--- Henrik Nordstrom <hno@squid-cache.org> wrote:
> Not sure what you are up to here. Can you explain a
> bit more?
>
> At a first glance is seems similar to ETag indexing
> where there may
> be several different content variants in one URL
> index position. See
> http://devel.squid-cache.org/ETag/ for how I did it
> there.
>
> Regards
> Henrik
>
> On Sunday 21 April 2002 00:30, Yee Man Chan wrote:
> > Thanks for your reply, Henrik. I was trying to the
> > URL-indexed StoreEntry with
> Content-Digest-indexed. So
> > I need to store Metadata that maps multiple URLs
> to a
> > content digest. This mapping is kind of dynamic
> > because the content at a particular URL may change
> and
> > then this URL will then map to a different content
> > digest. Do you have any recommendation on how I
> should
> > store these mappings?
> >
> > Thanks a lot.
> > Yee Man
> >
> > --- Henrik Nordstrom <hno@squid-cache.org> wrote:
> > > No.
> > >
> > > In the long run this will be up to each store
> > > implementation to deal
> > > with how the data gets stored.. along with how
> to
> > > deal with metadata
> > > and header updates.. but the store API needs a
> few
> > > revisions before
> > > we are fully there.
> > >
> > >
> > > Regards
> > > Henrik
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Yahoo! Games - play chess, backgammon, pool and
> more
> > http://games.yahoo.com/

__________________________________________________
Do You Yahoo!?
Yahoo! Games - play chess, backgammon, pool and more
http://games.yahoo.com/
Received on Sun Apr 21 2002 - 07:26:44 MDT

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:15:16 MST