Re: [squid-users] error "queue overload. Request rejected

Vipin Kaimal wrote:
> Hi All,
>
> Recently we installed a new squid proxy (squid-2.5.STABLE12-18.9 on
> SuSe 10) with LDAP authentication. Earleier we had a squid proxy with
> ldap authentication. That time, both LDAP and squid were running on
> the same server and everything was fine. Now we installed a new linux
> server and moved the squid service to it. LDAP is running on a
> different server. Many users are now complaining that they are not
> able to browse. Ever since I migrated all users to this new proxy
> server, we are getting the following errors in /var/log/message file.
> The same messages are displayed in /var/log/squid/cache.log
>
> Feb 17 11:32:21 proxy squid[9869]: aclMatchExternal: 'fullaccess'
> queue overload. Request rejected.
> Feb 17 11:32:21 proxy squid[9869]: aclMatchExternal:
> 'restrictedaccess' queue overload. Request rejected.
> Feb 17 11:32:21 proxy squid[9869]: aclMatchExternal: 'deniedaccess'
> queue overload. Request rejected.
> Feb 17 11:32:21 proxy squid[9869]: aclMatchExternal: 'fullaccess'
> queue overload. Request rejected.
> Feb 17 11:32:21 proxy squid[9869]: aclMatchExternal:
> 'restrictedaccess' queue overload. Request rejected.
> Feb 17 11:32:21 proxy squid[9869]: aclMatchExternal: 'deniedaccess'
> queue overload. Request rejected.
> Feb 17 11:32:21 proxy squid[9869]: aclMatchExternal: 'fullaccess'
> queue overload. Request rejected.
> Feb 17 11:32:21 proxy squid[9869]: aclMatchExternal:
> 'restrictedaccess' queue overload. Request rejected.
> Feb 17 11:32:21 proxy squid[9869]: aclMatchExternal: 'deniedaccess'
> queue overload. Request rejected.
> Feb 17 11:32:21 proxy squid[9869]: aclMatchExternal: 'fullaccess'
> queue overload. Request rejected.
>
> In cache.log these lines are also displayed.
> WARNING: All deniedaccess processes are busy.
> 2009/02/17 11:34:40| WARNING: up to 5 pending requests queued
> 2009/02/17 11:34:40| Consider increasing the number of deniedaccess
> processes to at least 10 in your config file.
> 2009/02/17 11:34:44| aclMatchExternal: 'fullaccess' queue overload.
> Request rejected.
> 2009/02/17 11:34:44| aclMatchExternal: 'restrictedaccess' queue
> overload. Request rejected.
> 2009/02/17 11:34:44| aclMatchExternal: 'deniedaccess' queue overload.
> Request rejected.
>
> Here is my external acl definition.
>
> #eDir full Internet access
> external_acl_type fullaccess %LOGIN /usr/sbin/squid_ldap_group -d -b
> "o=org" -B "o=org" -D "cn=squidproxy,ou=services,o=org" -w "" -s sub
> -h 10.10.10.3 -f "(&(objectclass=groupOfNames)(cn=%a))" -F
> "(&(&(objectClass=Person)(cn=%s))(groupMembership=cn=proxyfullaccess,ou=groups,ou=services,o=org))"
> -Z
>
> #eDir deny Internet access
> external_acl_type deniedaccess %LOGIN /usr/sbin/squid_ldap_group -d -b
> "o=org" -B "o=org" -D "cn=squidproxy,ou=services,o=org" -w "pass" -s
> sub -h 10.10.10.3 -f "(&(objectclass=groupOfNames)(cn=%a))" -F
> "(&(&(objectClass=Person)(cn=%s))(groupMembership=cn=proxydenyaccess,ou=groups,ou=services,o=org))"
> -Z
>
> #eDir limited Internet access
> external_acl_type restrictedaccess %LOGIN /usr/sbin/squid_ldap_group
> -d -b "o=org" -B "o=org" -D "cn=squidproxy,ou=services,o=org" -w
> "pass" -s sub -h 10.10.10.3 -f "(&(objectclass=groupOfNames)(cn=%a))"
> -F "(&(&(objectClass=Person)(cn=%s))(groupMembership=cn=proxylimitaccess,ou=groups,ou=services,o=org))"
> -Z
>
>
> This was the same configuration we had in our earlier setup and it was
> working fine. Please suggest me what should be the root cause for this
> and help me in resolving this.
>
> Thanks in Advance,
>
> Vinukaimal

The new remote LDAP server has additional network lag time. It is now
far too slow for the auth load you are placing on it. Squid is
overloading the helpers waiting queues.

Your options are to:
  - improve the network link between the machines
  - increase the helper queue size, (add children=N just before the
%LOGIN). N being >=10. (maybe concurrency=10 will work too, I'm not
sure, if it does thats better).
  - increase the result caching time (add ttl=X and negative_ttl=X
before the %LOGIN). X being seconds for delay between re-checks.
  - all of the above.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE13
   Current Beta Squid 3.1.0.5