Re: [squid-users] Peering caches (squid and 3rd parties) - How to

On 12/06/2013 8:24 a.m., Guillermo Javier Nardoni - Grupo GERYON wrote:
> Hello everyone,
>
> We have this situation and we tried a lot of configurations without success.
>
> • 1000 Customers
> • 4 Caches BOX running Squid 2.7 on Debian Squeeze • Caches are full-meshed
> to each other • Every Squid is running in transparent mode (http_port 3128
> transparent) • Every Squid is running HAARPCACHE on localhost at port 8080
> (HAARPCACHE is a Thundercache 3.1 fork wich Works PERFECT for caching sites
> like youtube with lots of HITS) .
> • Every Squid is connected to Internet through RB1 • RB2 (Mikrotik RouterOS)
> is doing round-robin selection on every squid redirecting all trafic to port
> 80 to internet to port 3128 on squid

<snip>

> As you can see, the same file is downloaded twice (at least) if the petition
> is not redirected to the same cache box.
> How can I achieve the goal to ask every cache and if the file is cached on
> any sibling or parent it shouldn’t be downloaded from internet but the cache
> itself.

The simplest thing you can do with your existing proxies is to set them
up into a CARP installation.

With the CARP design you have two layers of proxies:
- layer 1 is the Squid acting as gateways between clients and wherever
the data comes from.
- layer 2 is the HAARPCACHE proxies acting as caches for that specific
content.

To change you current configuration into a CARP system all you need to
do is:

1) make all HAARP proxies listen on IP:port which are accessible from
any of the Squid.

2) add a cache_peer line to each Squid.conf pointing at each HAARP proxy.
+ Using the "carp" option on every one of these cache_peer lines.
+ Use the same cache_peer_access ACL setup that you have now, but for
everyone of those new cache_peer as well.

3) ensure each of your Squid always have identical cache_peer settings.
- you can do that by writing all the HAARP related settings into a
separate file which is mirrored between the Squid and using squid.conf
include directive to load it.

After this no matter which squid receives the client request it will
hash the URL and point it at the HAARP proxy which is most likely to
have already cached it.

Done.

Extra notes:

* in a textbook CARP array all requests go to the parent caches - this
is optional. In your case only the HAARP URLs will go there.

* in a textbook CARP array the frontend does not cache - this is optional.

Amos
Received on Wed Jun 12 2013 - 06:33:29 MDT