Re: [squid-users] NTLM auth and ContentLength = 0

On 11/08/11 00:04, Christian Gregoire wrote:
> Hello,
>
> I use Squid 3.1.9 + ICAP + ClamAV with NTLM authentication on a CentOS box. It
> works pretty well, except in one particular case.
>
> Here, the HTTP client is a third-party software on Windows, not a standard
> navigator, which makes a few HTTP requests when it is launched.
>
> Most of the requests show the NTLM challenge/response steps correctly, but not
> the last one which is denied by the Squid service. The only special thing I can
> see is that the content length of that request is set to zero (see the traces
> and the headers below).

Maybe. I recall some talk about 0-length POST a while back. But there
have been no patches related to it submitted yet.

I also notice that the failed attempt has a much longer blob tag than
the successful one.

Check cache.log for any mentions of problems. Perhapse enable debugging
with -d on the helper to see if there is an issue with the validation.

>
> Please note: if NTLM auth is disabled on the Squid server, it works fine.
>
>
> 1312956350.701 0 10.1.100.5 TCP_DENIED/407 3837 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html
> 1312956350.702 0 10.1.100.5 TCP_DENIED/407 4219 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html
> 1312956351.543 841 10.1.100.5 TCP_MISS/200 721 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet
> expinet.colissimo DIRECT/84.37.93.36 text/xml
> 1312956351.559 0 10.1.100.5 TCP_DENIED/407 3837 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html
> 1312956351.560 0 10.1.100.5 TCP_DENIED/407 4219 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html
> 1312956352.390 830 10.1.100.5 TCP_MISS/200 720 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet
> expinet.colissimo DIRECT/84.37.93.36 text/xml
> 1312956352.407 0 10.1.100.5 TCP_DENIED/407 3837 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html
> 1312956352.408 0 10.1.100.5 TCP_DENIED/407 4219 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html
> 1312956353.281 873 10.1.100.5 TCP_MISS/200 716 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet
> expinet.colissimo DIRECT/84.37.93.36 text/xml
> 1312956353.296 0 10.1.100.5 TCP_DENIED/407 3837 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html
> 1312956353.298 0 10.1.100.5 TCP_DENIED/407 4219 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html
> 1312956354.165 868 10.1.100.5 TCP_MISS/200 715 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet
> expinet.colissimo DIRECT/84.37.93.36 text/xml
> 1312956354.189 0 10.1.100.5 TCP_DENIED/407 3845 POST
> http://www.colis-logistique.com/expeditor/updateApplication/servlet - NONE/-
> text/html
> 1312956354.190 0 10.1.100.5 TCP_DENIED/407 4227 POST
> http://www.colis-logistique.com/expeditor/updateApplication/servlet - NONE/-
> text/html
> 1312956355.005 814 10.1.100.5 TCP_MISS/200 719 POST
> http://www.colis-logistique.com/expeditor/updateApplication/servlet
> expinet.colissimo DIRECT/84.37.93.36 text/xml
> 1312956355.016 0 10.1.100.5 TCP_DENIED/407 3773 GET
> http://www.colis-logistique.com/updatesite? - NONE/- text/html
> 1312956355.017 0 10.1.100.5 TCP_DENIED/407 4155 GET
> http://www.colis-logistique.com/updatesite? - NONE/- text/html
> 1312956355.579 561 10.1.100.5 TCP_MISS/200 765 GET
> http://www.colis-logistique.com/updatesite? expinet.colissimo DIRECT/84.37.93.36
> APPLICATION/OCTET-STREAM
> 1312956356.570 430 10.1.100.5 TCP_MISS/200 4599 POST
> http://www.colis-logistique.com/expeditor/updateaccount/servlet
> expinet.colissimo DIRECT/84.37.93.36 text/xml
> 1312956357.437 769 10.1.100.5 TCP_MISS/200 720 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet
> expinet.colissimo DIRECT/84.37.93.36 text/xml
> 1312956357.452 0 10.1.100.5 TCP_DENIED/407 3837 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html
> 1312956357.454 0 10.1.100.5 TCP_DENIED/407 4219 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html
> 1312956358.267 814 10.1.100.5 TCP_MISS/200 715 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet
> expinet.colissimo DIRECT/84.37.93.36 text/xml
> 1312956359.448 0 10.1.100.5 TCP_DENIED/407 3835 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html<---- STEP 1
> 1312956359.449 0 10.1.100.5 TCP_DENIED/407 4217 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html<---- STEP 2
> 1312956359.451 0 10.1.100.5 TCP_DENIED/407 4193 POST
> http://www.colis-logistique.com/expeditor/updateReference/servlet - NONE/-
> text/html<---- STILL DENIED !!!!!!
>
> ------------------- Headers of the HTTP session for the denied request :
>
> POST http://www.colis-logistique.com/expeditor/updateReference/servlet HTTP/1.1
> Content-Type: application/x-www-form-urlencoded
> User-Agent: TELINTRANSCOM
> Host: www.colis-logistique.com
> Content-Length: 0
> Pragma: no-cache
>
> HTTP/1.0 407 Proxy Authentication Required
> Server: squid/3.1.9
> Mime-Version: 1.0
> Date: Wed, 10 Aug 2011 11:41:16 GMT
> Content-Type: text/html
> Content-Length: 3469
> X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
> Vary: Accept-Language
> Content-Language: en
> Proxy-Authenticate: NTLM
> X-Cache: MISS from fw-master
> Via: 1.0 fw-master (squid/3.1.9)
> Connection: close
>
> POST http://www.colis-logistique.com/expeditor/updateReference/servlet HTTP/1.1
> Content-Type: application/x-www-form-urlencoded
> User-Agent: TELINTRANSCOM
> Host: www.colis-logistique.com
> Content-Length: 0
> Pragma: no-cache
> Proxy-Connection: Keep-Alive
> Proxy-Authorization: NTLM
> TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAFAs4OAAAADw==
>

What application is this? there are two bugs in those headers that need
reporting. Not related to your NTLM problems though.
  "TELINTRANSCOM" appears to be a company name rather than a software
product name so I have no way to contact them myself to do it.

  * Pragma: no-cache only works (sometimes) for HTTP/1.0 software, and
should not be sent without a matching Cache-Control: no-cache for
HTTP/1.1 softwares.

  * Proxy-Connection: is not a correct header. It should be just Connection:

> HTTP/1.0 407 Proxy Authentication Required
> Server: squid/3.1.9
> Mime-Version: 1.0
> Date: Wed, 10 Aug 2011 11:41:16 GMT
> Content-Type: text/html
> Content-Length: 3605
> X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
> Vary: Accept-Language
> Content-Language: en
> Proxy-Authenticate: NTLM
> TlRMTVNTUAACAAAADAAMADAAAAAFgomifYF1+R0dG4gAAAAAAAAAAHYAdgA8AAAAUABJAEMASABPAE4AAgAMAFAASQBDAEgATwBOAAEAEgBGAFcALQBNAEEAUwBUAEUAUgAEABgAcABpAGMAaABvAG4ALgBsAG8AYwBhAGwAAwAsAGYAdwAtAG0AYQBzAHQAZQByAC4AcABpAGMAaABvAG4ALgBsAG8AYwBhAGwAAAAAAA==
>
> X-Cache: MISS from fw-master
> Via: 1.0 fw-master (squid/3.1.9)
> Connection: keep-alive
>
> POST http://www.colis-logistique.com/expeditor/updateReference/servlet HTTP/1.1
> Content-Type: application/x-www-form-urlencoded
> User-Agent: TELINTRANSCOM
> Host: www.colis-logistique.com
> Content-Length: 0
> Pragma: no-cache
> Proxy-Connection: Keep-Alive
> Proxy-Authorization: NTLM
> TlRMTVNTUAADAAAAGAAYAKIAAAAYABgAugAAAAwADABIAAAARgBGAFQAAAAIAAgAmgAAAAAAAADSAAAABYKIogUCzg4AAAAPUABJAEMASABPAE4AZQB4AHAAaQBuAGUAdAAuAGMAbwBsAGkAcwBzAGkAbQBvACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgAFQAUwBFADEAFnMeVH6eNxAAAAAAAAAAAAAAAAAAAAAAEueFV9XBLGkb2/4/sGwqnNiuOXFXC5lA
>
>
> HTTP/1.0 407 Proxy Authentication Required
> Server: squid/3.1.9
> Mime-Version: 1.0
> Date: Wed, 10 Aug 2011 11:41:16 GMT
> Content-Type: text/html
> Content-Length: 3829
> X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
> Vary: Accept-Language
> Content-Language: en
> Proxy-Authenticate: NTLM
> X-Cache: MISS from fw-master
> Via: 1.0 fw-master (squid/3.1.9)
> Connection: close
>
> ------------------- Headers for an accepted one :
>
> POST http://www.colis-logistique.com/expeditor/updateReference/servlet HTTP/1.1
> Content-Type: application/x-www-form-urlencoded
> User-Agent: TELINTRANSCOM
> Host: www.colis-logistique.com
> Content-Length: 587
> Pragma: no-cache
>
> HTTP/1.0 407 Proxy Authentication Required
> Server: squid/3.1.9
> Mime-Version: 1.0
> Date: Wed, 10 Aug 2011 11:40:40 GMT
> Content-Type: text/html
> Content-Length: 3471
> X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
> Vary: Accept-Language
> Content-Language: en
> Proxy-Authenticate: NTLM
> X-Cache: MISS from fw-master
> Via: 1.0 fw-master (squid/3.1.9)
> Connection: close
>
> POST http://www.colis-logistique.com/expeditor/updateReference/servlet HTTP/1.1
> Content-Type: application/x-www-form-urlencoded
> User-Agent: TELINTRANSCOM
> Host: www.colis-logistique.com
> Content-Length: 587
> Pragma: no-cache
> Proxy-Connection: Keep-Alive
> Proxy-Authorization: NTLM
> TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAFAs4OAAAADw==
>
> HTTP/1.0 407 Proxy Authentication Required
> Server: squid/3.1.9
> Mime-Version: 1.0
> Date: Wed, 10 Aug 2011 11:40:40 GMT
> Content-Type: text/html
> Content-Length: 3607
> X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
> Vary: Accept-Language
> Content-Language: en
> Proxy-Authenticate: NTLM
> TlRMTVNTUAACAAAADAAMADAAAAAFgomiNP/Vxjp4/tAAAAAAAAAAAHYAdgA8AAAAUABJAEMASABPAE4AAgAMAFAASQBDAEgATwBOAAEAEgBGAFcALQBNAEEAUwBUAEUAUgAEABgAcABpAGMAaABvAG4ALgBsAG8AYwBhAGwAAwAsAGYAdwAtAG0AYQBzAHQAZQByAC4AcABpAGMAaABvAG4ALgBsAG8AYwBhAGwAAAAAAA==
>
> X-Cache: MISS from fw-master
> Via: 1.0 fw-master (squid/3.1.9)
> Connection: keep-alive
>
> POST http://www.colis-logistique.com/expeditor/updateReference/servlet HTTP/1.1
> Content-Type: application/x-www-form-urlencoded
> User-Agent: TELINTRANSCOM
> Host: www.colis-logistique.com
> Content-Length: 587
> Pragma: no-cache
> Proxy-Connection: Keep-Alive
> Proxy-Authorization: NTLM
> TlRMTVNTUAADAAAAGAAYAH4AAAAYABgAlgAAAAwADABIAAAAIgAiAFQAAAAIAAgAdgAAAAAAAACuAAAABYKIogUCzg4AAAAPUABJAEMASABPAE4AZQB4AHAAaQBuAGUAdAAuAGMAbwBsAGkAcwBzAGkAbQBvAFQAUwBFADEAuyOcnxnMyogAAAAAAAAAAAAAAAAAAAAAGvDfkb4KZM8Lkgec9ot0QL5qpUrN+xaa
>
>
> HTTP/1.0 200 OK
> Date: Wed, 10 Aug 2011 11:34:59 GMT
> Server: Apache
> Vary: User-Agent
> Content-Type: text/xml
> X-Cache: MISS from fw-master
> Via: ICAP/1.0 fw-master.domain.local (C-ICAP/0.1.3 SquidClamav/Antivirus service
> ), 1.0 fw-master (squid/3.1.9)
> Connection: close
>
> ------------------- Squid configuration file :
>
> http_port 3129
> cache_access_log /servers/squid/logs/access.log
> cache_store_log /servers/squid/logs/store.log
> icap_enable on
> icap_send_client_ip on
> icap_send_client_username on
> icap_client_username_encode off
> icap_client_username_header X-Authenticated-User
> icap_preview_enable on
> icap_preview_size 1024
> icap_service service_req reqmod_precache bypass=1
> icap://127.0.0.1:1344/squidclamav
> adaptation_access service_req allow all
> icap_service service_resp respmod_precache bypass=1
> icap://127.0.0.1:1344/squidclamav
> adaptation_access service_resp allow all
>
> auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp
> auth_param ntlm children 5
> external_acl_type GroupeInternet %LOGIN /usr/local/squid/libexec/wbinfo_group.pl
>
> acl AccesInternetOK external GroupeInternet gg_internet
> acl CONNECT method CONNECT
>
> http_access allow CONNECT

Sigh. Your proxy has no security. NTLM is an illusion.

Try this:
   squidclient -P request.txt -m CONNECT google.com:80

Where request.txt contains:
"
GET / HTTP/1.1
Host: google.com

"
Poof. No login.

This is why Safe_ports and SSL_Ports exists. Please use them.

> http_access allow AccesInternetOK
> http_access deny all
>
>
>
> Any idea ?
>
> Christian

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.14
   Beta testers wanted for 3.2.0.10