RE: [squid-users] ICAP Bypassing Causing Performance Issues

Thanks Amos - regex pattern we're using is:

.*some_url_end.html$

We also have many individual domains which we're bypassing

acl bypassIcapRequest dstdomain "/apps/cwapps/squid-3/etc/byPass_ICAP_request.properties"
icap_access class_1 deny bypassIcapRequest

as time has gone on - we've been adding more URLs to this list also (currently up to 39 URLs) - this won't be doing regular expression matching, but we've seen as time goes on, more and more established connections on ICAP server port. Also CPU usage going up, and we're seeing more 'essential ICAP service is down' errors in the logs.

Traffic has not changed significantly - in fact has maybe gone down. The only change we can really identify is the extra bypassed domains.

Does squid parse the properties file for every hit?

Also, we've only been reconfiguring squid when we update this file. Is this enough, or do we need to restart?

Will look into extra debugging now.

Thanks and regards,
Justin

-----Original Message-----
From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
Sent: Monday, August 22, 2011 10:29 PM
To: squid-users_at_squid-cache.org
Subject: Re: [squid-users] ICAP Bypassing Causing Performance Issues

On 23/08/11 00:03, Justin Lawler wrote:
> Hi,
>
> We have had to put in a number of URLs to the squid bypass
>
> icap_service service_1 reqmod_precache 0 icap://127.0.0.1:1344/reqmod
> icap_class class_1 service_1
>
> acl bypassIcapRequestURLregex urlpath_regex "./squid-3/etc/byPass_ICAP_request_URLregex.properties"
> icap_access class_1 deny bypassIcapRequestURLregex
>
>
> When we added 4 regular expressions to this file, we started to see the CPU usage going up quite a bit, and we started to see the number of established connections from squid to ICAP server double or triple.
>
> Is this a known issue? Is there a better/more efficient way to bypass ICAP than above?

Other than using other ACL types, no.

>
> Regular expressions were very simple, just matching end of URLs.

a) regex is a bit slow. Did you remember to anchor the ends? and
manually aggregate the patterns? avoid extended-regex pattern tricks?

b) URLs can be many KB in length. That can make URL regex very CPU
intensive.

d) routing selection ACLs are run multiple times per request.

You can turn on access control debugging (level 28,3) to see how many
times those are run and how long they take each test.

>
> We're running squid 3.0.15 on Solaris 10.
>

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.14
   Beta testers wanted for 3.2.0.10
This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,
you may review at http://www.amdocs.com/email_disclaimer.asp