RE: [squid-users] Too many queued ntlmauthenticator requests and squid failed

Hi Guys,

This is my http_access sequence, but iam using redirector_access deny at the top, can it cause too may ntlmauthenticator requests ?

Please have a look and let me know your thoughts,


auth_param ntlm program /usr/local/squid/libexec/ntlm_auth --helper-protocol=squid-2.5-ntlmssp
auth_param ntlm children 25
auth_param basic program /usr/local/squid/libexec/ntlm_auth --helper-protocol=squid-2.5-basic
auth_param basic children 10
auth_param basic realm xxxxxxxxxxxx
auth_param basic credentialsttl 2 hours
auth_param basic casesensitive off
authenticate_cache_garbage_interval 10 seconds
authenticate_ttl 0 seconds


external_acl_type nt_group ttl=0 children=25 %LOGIN /usr/local/squid/libexec/wbinfo_group.pl
acl AuthorizedUsers proxy_auth REQUIRED
acl NoRedirect external nt_group ProxyUnrestAccess
acl youtube_domains dstdomain .youtube.com
acl utubeallow external nt_group ProxyYouTubeAccess
acl restrictedusers external nt_group proxyaccessfull



redirecotry_access deny Noredirect
http_access allow utubeallow
http_access deny youtube_domains
http_access allow AuthorizedUsers restrictedusers
http_access allow localnet AuthorizedUsers restrictedusers
http_access deny all


Thanks for your time.

Tharanga




-----Original Message-----
From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
Sent: Tuesday, 19 October 2010 10:15 AM
To: Tharanga Abeyseela (RGA)
Cc: squid-users_at_squid-cache.org
Subject: RE: [squid-users] Too many queued ntlmauthenticator requests and squid failed

On Mon, 18 Oct 2010 22:54:42 +0000, "Tharanga Abeyseela (RGA)" wrote:
> Hi,
>
> I tried to start squid in the offpeak time there were only 35 users, but
> still its says Too many queued ntlmauthenticator requests, and terminate
> the squid process.
>
> My domain is connected to several other domains (trusted domains) can it
> cause this issue ?
>
>
> Following are the error messages on log files.
>
> Squid Parent: child process 15206 exited with status 1
> Oct 18 19:47:29 rapsydprox2 squid[15204]: Squid Parent: child process
> 20806 exited due to signal 15
>
> 2010/10/18 19:47:10, 2] winbindd/winbindd.c:872(remove_client)
> final write to client failed: Broken pipe
> [2010/10/18 19:47:10, 0]
> winbindd/winbindd.c:954(winbindd_listen_fde_handler)
> winbindd: Exceeding 200 client connections, no idle connection found

Ouch. So your problem then is that your Squid receives greater than *200*
concurrent *new* TCP connections (per second or so) with only 35 users.
This is about double the maximum number of concurrent connections modern
web browser make to proxies (4-6).
 * Make sure that your squid.conf has at least
client_persistent_connections and server_persistent_connections turned on.
It might reduce this a little bit.
 * check your squid.conf http_access listing order. Make as many
rejections as possible before testing for auth. This will usually have a
chain reaction down to increased average service times.

Note that squid-3.0 does not have "connection pinning" which is required
to connect to anything on the other side of Squid with NTLM. Connections
*to* Squid with NTLM only required persistent connections.

I really strongly urge you move to 3.1.8. We have done a lot of work to
make connections more persistent since 3.0 was released.


> [2010/10/18 19:47:21, 0]
> winbindd/winbindd.c:954(winbindd_listen_fde_handler)
> winbindd: Exceeding 200 client connections, no idle connection found
> [2010/10/18 19:47:21, 2] winbindd/winbindd.c:872(remove_client)
> final write to client failed: Broken pipe
>
>
> 5 Oct 18 16:58:37 rapsydprox2 (squid): Too many queued
ntlmauthenticator
> requests
> 976 Oct 18 17:00:23 rapsydprox2 (ntlm_auth): [2010/10/18 17:00:23,
0]
> utils/ntlm_au
> 977 Oct 18 17:00:23 rapsydprox2 (ntlm_auth): could not obtain
winbind
> domain name!
> Following is my smb.conf
>
> workgroup = zzz
> realm = zzz.COM
> password server = *
> #netbios name = zzz
> server string = Samba file and print server
> security = ADS
> encrypt passwords = yes
> log level = 2
> log file = /var/log/samba/%m
> max log size = 50
> winbind separator = \.
> #printcap name = cups
> #printing = cups
> ##new
> idmap cache time = 1800
> idmap negative cache time = 300
> winbind cache time = 1800
> ##end new
> idmap uid = 15000-20000
> idmap gid = 15000-20000
> #winbind use default domain = yes
> nt acl support = yes
> map acl inherit = yes
> winbind enum users = yes
> winbind enum groups = yes
> #client ntlmv2 auth = yes
> template homedir = /home/%D/%U
> template shell = /bin/bash
> ldap passwd sync = yes
>
>
>
> Appreciate your help on this.
>
> Thanks,
>
> Tharanga
>
>
> -----Original Message-----
> From: Amos Jeffries
>
> On Mon, 18 Oct 2010 00:50:28 +0000, "Tharanga Abeyseela \(RGA\)" wrote:
>> Hi Guys,
>>
>> I have been running squid with AD authentication and security group
>> authent= ication for the last 6 months, and suddenly squid failed with
> the
>> following= message
>>
>> (squid): Too many queued ntlmauthenticator requests.
>
> Sign of a sudden burst of people logging in. Or that something caused a
> slightly higher than normal delay in the auth checks. Which multiplies
out
> to fill up the Squid helpers.
>
>>
>> Iam using
>>
>> samba-3.4.8-59.fc12.x86_64
>> samba-winbind-clients-3.4.8-59.fc12.x86_64
>> samba-common-3.4.8-59.fc12.x86_64
>> samba-client-3.4.8-59.fc12.x86_64
>> samba-winbind-devel-3.4.8-59.fc12.x86_64
>> samba-winbind-3.4.8-59.fc12.x86_64
>> Squid Cache: Version 3.0.STABLE24
>> configure options: '--prefix=3D/usr/local/squid'
>> '--enable-auth=3Dntlm,bas= ic'
>> '--enable-external-acl-helpers=3Dwbinfo_group'
>>
>> and also iam getting the following error
>>
>> [2010/10/18 10:28:58, 3] winbindd/winbindd_cm.c:1597(connection_ok)
>> connection_ok: Connection to for domain xxx is not connected
>>
>> (but if I use the net join command it says joined to domain)
>>
>> According to my squid report normally there are 700 people connecting
to
>> th= e proxy, but now its says winbind maximum connection exceeds 200,
> but
>> iam = confusing about that.
>>
>> Appreciate your help on this.
>>
>> Best regards,
>> Tharanga
>
> Two basic design flaws in NTLM:
> - winbind cannot accept more than ~256 simultaneous stateful logins (ie
> NTLM)
> - NTLM requires every new TCP connection to perform a full 2-request
> handshake
> (ie minimum login delays of request+reply+request+reply in HTTP)
>
> There are two things which each reduce the problem a little. When
combined
> they provide a great improvement in network speed and logins...
>
> - upgrade to 3.1.8.
> - upgrade to Negotiate/Kerberos from NTLM
>
>
> Amos
Received on Thu Oct 21 2010 - 00:07:33 MDT