Re: [squid-users] Optimized Squids

On Wed, 23 Jun 2010 09:51:40 -0500, Seann Clark
<nombrandue_at_tsukinokage.net> wrote:
> Amos Jeffries wrote:
>> On Tue, 22 Jun 2010 16:05:04 -0500, Seann Clark
>> <nombrandue_at_tsukinokage.net> wrote:
>>
>>> All,
>>>
>>> I have been playing with/tweaking/breaking my squid for a few
months
>>>
>>
>>
>>> now, and I am looking for suggestions from the list on improving
>>> performance. This is on a home system, which does not have a large
user
>>> base. I am running a Dual Xeon 2.0 Ghz system with 2 gigs ram, 120
Gigs
>>> hard drive, in a Raid 5 configuration controlled by a 3ware RAID card.
>>>
>>
>> Optimization Tip #1: get rid of the RAID-5.
>> http://wiki.squid-cache.org/SquidFaq/RAID
>>
>>
> I am planning on adding in a new disk to the unit, to move the cache off

> the RAID volume, and keep the RAID volume just for the Operating system.

> Would that have any impact on squid if its binary lives on the RAID-5
> volume, but the caches are on separate physical drives?

It would get faster. That is all.

You can even do it live for no downtime:
 update squid.conf with new cache_dir paths
 run "squid -z"
 run "squid -k reconfigure" to shift the cache location in the running
copy.
 remove old RAID directories.

>>> I was using the stock Fedora 8 RPM for this, which was single
threaded,
>>> squid 2.6 Stable 22. I am also running this with diskd currently.
>>>
>>
>> Optimization tip #2: run the fastest Squid release.
>> Today 2.7 is current fastest official for production use.
>> If you are willing to run semi-production code 2.HEAD daily snapshots
>> have more speed experiments and some few high-performance testers
behind
>> it.
>>
>>
>>> I have recently recompiled squid to the latest stable for version
>>> 2.7 (Stable 9) with the async io flag passed to the configure command.

>>> After a little updating of my configurations, just enough to get it to

>>> work (I haven't changed any of the settings that are new to 2.7, so
they
>>>
>>
>>
>>> are defaults right now), I have noticed a drastic improvement in speed

>>> and even when the system is at a high load (3-5 on the system load,
>>>
>>
>> So that covers #2. :)
>>
>>
>> Optimization tip #3: use fastest disk IO method available.
>> Fedora being a linux that would be AUFS.
>>
>> Specifically in 2.7, AUFS tuned via cache_dir directive for large
>> objects
>> (~1MB or more) alongside COSS directories for objects smaller than the
>> AUFS
>> holds.
>> NP: these two caches types are happy to share one disk between them. An
>> exception to the on-disk-per-cache_dir rule of thumb.
>>
>>
>>
> To get this a little clearer, as I am still learning what I can do with
> the cache, I would configure the directives as such (example sizes):
>
> cache_dir coss /var/spool/coss_cache 1024 1
> cache_dir aufs /var/spool/bigcache 1024 100

There are three numbers needed (size in MB, number of top level
directories, number of sub-directories)

The options after those for AUFS you will need the min-size=N parameter
and the COSS dir will need the max-size=N parameter (non-overlapping so one
byte difference)

http://www.squid-cache.org/Versions/v2/2.7/cfgman/cache_dir.html
has full documentation on the directive and its settings. The specific
relevant bits you want to read are under "The coss store type:" and "Common
options:" at the bottom.

>
>>> reported by top) it runs fairly well. I am looking at moving my cache
>>> directories off to a different disk, formatted with reiserFS, I am
also
>>> planning on running it using diskd as the storage scheme. The drive I
am
>>>
>>
>>
>> see tip #3. AUFS makes use of multi-core threads, diskd process is
>> single-threaded even if it runs additional to the Squid main thread.
>>
>> From my own experience, Squid has had no direct benefit from RieserFS
>> over
>> extN. The OS and other apps sometimes do gain, so it's up to you.
>>
>> You will need to tune the items which get journalled such as noatime
etc
>> to increase disk IO speeds either way.
>>
>>
>>
>>> using will be about 100 gigs of formatted space, and I plan to use all

>>> that space for the cache.
>>>
>>
>> Make sure you have enough RAM for the indexes. 2GB is borderline for
your
>> planned cache size.
>> 4KB per object stored is what the index uses, with up to 2^31 objects
per
>> cache_dir = up to 8GB index. Reality varies with your users habits.
>>
>> NP: 2.7 in-memory handling of large objects is not great, so
sacrificing
>> any large cache_mem RAM cache for COSS small-object cache is probably a
>> good trade-off in that release.
>>
>>
> Expanded RAM in the system is planned, so it is a good point you
> mentioned as it gives me a baseline to get for the system
>>> I would like to know if this is a good plan, or should I change
some
>>>
>>
>>
>>> things and how to change them, as well as any suggestions of
>>> configuration settings with the cache, and 2.7 options that may make a

>>> cache run even faster.
>>>
>>>
>>
>> On top of those basic parameters, your ACL configuration strongly
>> influences speed. Read the wiki ACL pages on which are grouped as
>> fast/slow
>> (NP: 'fast' does not strictly mean speed there) to get the ordering
>> optimal.
>> Avoid any operations using regex like plague.
>> Avoid helper operations as well since they often introduce large
amounts
>> of lag as well.
>>
>> Amos
>>
> I don't use regex in my configuration, and all my ACL's are set for
> default access for snmp, cache access, and cachemanager access.
>
> ~Seann
Received on Thu Jun 24 2010 - 06:21:28 MDT