fre 2013-07-26 klockan 14:44 +0300 skrev Eliezer Croitoru:
> So etag is fine but per server or per url.
ETag is per URL.
> metalink\hashes are more global.
Yes.
> since the list of mirrors is big and there is a list of them it would be
> very simple to use StoreID helper to identify them faster then a HASH
> verification.
Hasing is required for a distributed system. You also need to worry
about people intentionally or by accident trying to inject false
material in the cache.
It is not uncommon that Fedora mirrors gets out of sync or even having
corrupted files. The application layer takes care of that by discarding
objects with wrong checksum and trying another mirror.
> The only problem is that there should be a computer\program friendly
> list of mirrors for let say fedora..
Yes...
> Once now squid do IMS then all the mirrors should have the same IMS
> response for all mirrors(if they use rsync).
IMS works in most cases, as mirrors usually preserve modification date.
But it is quite weak as cache validation.
> the public list of fedora contains currently 152 active public mirrors
> with 240Gbic/sec which a squid forward proxy can pretty easily cache all
> of the objects in one repo with basic settings.
Yes, the Fedora repo is not overly large. Even if you add Fedora,
Debian, Ubuntu, CentOS, Arch, Mageia etc it's still manageable in size.
> Do we have friends there??
Yes.
> I was thinking before to use a specific mealink file they have on
> specific resources to just allow the admin update the MIRROR list and
> then squid will know about all the mirrors.
Fedora have very easily parseable official mirror lists.
And there is a substantial amount of inofficial mirror sites.
> Notice that as we publish this feature we go from "hacking youtube" to
> "allowing admins to prevent de-duplication of objects DB".
Good.
Regards
Henrik
Received on Fri Jul 26 2013 - 18:23:12 MDT
This archive was generated by hypermail 2.2.0 : Sat Jul 27 2013 - 12:00:50 MDT