[squid-users] RAID is good (was: Re: [squid-users] Hardware setup ?)

I wish that the wiki for RIAD is rewritten.

Companies depend on internet access and a working Squid proxy
and therefore the advocated "no problem if a single disk fails"
is not from today's reality.

One should also consider the difference between
simple RAID and extremely advanced RAID disk systems
(i.e. EMC and other arrays).
The external disk arrays like EMC with internal RAID5 are simply faster
than a JBOD of internal disks.

Marcus

Amos Jeffries wrote:
> Michael Gale wrote:
>> Hey,
>>
>> We are working on our hardware requirements and am looking for
>> some feedback. Please let me know what you think:
>>
>> Demand:
>> - 225 requests per second during peak times in 2008. So we are
>> plaining for 300 RPS minimal per server. Ideally if each server could
>> handle 600 RPS that would be good.
>
> Squid 3.0 has up to 650rps
> Squid 2.6 has up to 850rps
>
> So either should easily match your requirements there.
>
>> - We have 1600 remote locations connected via sat link, each with
>> about 4 devices behind it.
>
> The satellite links will probably benefit from the collapsed-forwarding
> feature in squid 2.6 at either end of the links.
>
>> - 125GB per month of HTTP traffic
>>
>> We currently are planing on two servers being available behind an LVS
>> router. These two servers will speak with a squid instance at each
>> location so some form of peering can be used.
>>
>> So I have the following questions:
>>
>> 1. Would there be any problem with squid running at each sat location
>> (1600) trying to use a peering method with squidpeer.domain.com IP
>> that is load balanced by an LVS router pointing to two squid servers ?
>
> No problem.
> Assuming that you have explicit configuration of each remote location
> you may also be able to do away with the LVS balancer by enabling the
> traffic measurements squid can perform (cmp, netdb, icp delay-times).
> That cuts one potential bottleneck out.
>
>>
>> 2. Does squid benefit from a dual core or quad core setup at all ?
>>
>
> That depends on your external processes. The core Squid process is
> single-threaded. But redirectors, ICAP, external ACL, etc can all work
> on any CPU the system has additional to the CPU running squid.
>
>> 3. How do these hardware requirements look, per server:
>> - 4 drives for squid cache, hardware raid stripped
>
> RAID is a seriously bad idea with squid.
> http://wiki.squid-cache.org/SquidFaq/RAID
>
>> - 4ms seek time, 73GB of space =~ 294GB of cache available
>> - Looking to use at least 150GB of cache per server
>> - 8GB of RAM
>
> Rule of thumb: size of cache_mem + (size of cache * 0.05) == RAM needed.
>
> Meaning squid will probably use at least 2GB of ram for its indexes.
> That should leave you 4GB of memory-cache (cache_mem)
>
>> - Two dual core or two quad core 3.0Ghz processors.
>
> As I said above it depends on the extras that will be running whether
> quad-core is worth it.
>
> SMP is on the map, but probably a year away at least for stable
> production use in squid. So the benefits from any developments there are
> quite long-term.
>
> If I was in your situation I'd go Squid 2.6stable19.
>
> Amos
Received on Tue Mar 25 2008 - 07:23:39 MDT