RE: Squid memory footprint (was: MemPools rewrite)

On 5 Nov 2000, at 11:53, Chemolli Francesco (USI) <ChemolliF@GruppoCredit.it> wrote:

> WRT [1]:
> Besides, if we get an ICP query and we hit, there is a reasonable
> chance that we will get shortly a full-fledged query for that same
> object. So keeping an in-core cache of the most recently
> ICP-queried objects will save us a fair percentage of the overhead.

 The case when http fetch follows ICP-hit isn't a case for worry.
 The problem is for cases when fetch doesn't follow, like ICP-miss.

 Imagine a setup: 10 caches, loadbalanced with L4 switch, seen as a
 single cache by clients. Each box takes 1/10th of the http load, but
 to avoid cache pollution when each cache contains about the same
 content, peering setup is needed between the caches, and ICP gives
 currently best results. Thus all caches will see 100% of request
 rate as ICP queries.
 So, if ICP is slow, you either loose the benefits of loadbalancing,
 as ICP servicing will dominate the cache load, or have to accept
 non-ICP setup where all caches have eventually about the same content.
 Hard choice.

 In cache clusters we have to address icp (as a term, be it Digests,
 ICP or something new), that allows us in a fast and lightweight way
 to determine if any peer has requested object. Current Digests is
 not a good solution, during 1 hour between digest exchanges, lots of
 potential hits may get routed to the "wrong" cache.

 If we can keep ICP lightweight by some "microdigests" of squidFS
 metadata, and this microdigest can be updated instantly, then we
 should be ok.

> -----Original Message-----

> > ICP - shouldn't ever touch disks to return hit/miss. 2 boxes with
> equal
> > load and average hitrate of 30% would see 3 times more ICP traffic
> than
> > actual fetches from peers. We'd rather not allow this to touch disks.
>
> Well, usually I do not consider ICP as an useable option, and take the
> freedom not to consider it's requirements when thinking about store
> implementation. It should be possible to implement ICP with reasonable
> performance if the normal cache operations can be implemented with good
> performance.
>
> [1]

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 Delfi Online
 Tel: 6501 731, Fax: 6501 708
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Mon Nov 06 2000 - 04:39:47 MST