Thank you for the reply. As requested, below are 2 captured TCP streams. The first 3 sections is 1 TCP stream with the proxy in the data path, and captured from each interface of client<->proxy<->remote_server (although can't capture from the remote server interface). The 4th section is the TCP stream without the proxy in the data path.
Does it appear that the remote server is receiving a request in a format that it cannot process, and is returning the "# Server Error" page to the proxy?
===============================================================
PACKET CAPTURE FROM CLIENT WITH SQUID IN DATA PATH (STREAM 1):
GET http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: mirrors.rpmfusion.org
User-Agent: lwp-request/2.07
HTTP/1.0 200 OK
Date: Tue, 28 Apr 2009 17:44:45 GMT
Server: Apache/2.2.11 (Fedora)
Content-Length: 15
Cache-Control: no-cache
Content-Type: text/plain
X-Cache: MISS from la-squid.twistbox.com
X-Cache-Lookup: MISS from la-squid.twistbox.com:3128
Via: 1.0 la-squid.twistbox.com (squid)
Proxy-Connection: close
# Server Error
===============================================================
PACKET CAPTURE FROM CLIENT-SIDE TCP PORT OF THE PROXY(STREAM 1):
GET http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: mirrors.rpmfusion.org
User-Agent: lwp-request/2.07
HTTP/1.0 200 OK
Date: Tue, 28 Apr 2009 17:44:45 GMT
Server: Apache/2.2.11 (Fedora)
Content-Length: 15
Cache-Control: no-cache
Content-Type: text/plain
X-Cache: MISS from la-squid.twistbox.com
X-Cache-Lookup: MISS from la-squid.twistbox.com:3128
Via: 1.0 la-squid.twistbox.com (squid)
Proxy-Connection: close
# Server Error
===============================================================
PACKET CAPTURE FROM EXTERNAL-SIDE TCP PORT OF THE PROXY(STREAM 1):
GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.0
Host: mirrors.rpmfusion.org
User-Agent: lwp-request/2.07
Via: 1.1 la-squid.twistbox.com (squid)
X-Forwarded-For: unknown
Cache-Control: max-age=259200
Connection: keep-alive
HTTP/1.1 200 OK
Date: Tue, 28 Apr 2009 17:44:45 GMT
Server: Apache/2.2.11 (Fedora)
Content-Length: 15
cache-control: no-cache
Keep-Alive: timeout=2, max=100
Connection: Keep-Alive
Content-Type: text/plain
# Server Error
===============================================================
PACKET CAPTURE FROM CLIENT _WITHOUT_ THE PROXY IN DATA PATH(STREAM 2):
GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: mirrors.rpmfusion.org
User-Agent: lwp-request/2.07
HTTP/1.1 200 OK
Date: Tue, 28 Apr 2009 17:44:49 GMT
Server: Apache/2.2.11 (Fedora)
Content-Length: 404
cache-control: no-cache
Connection: close
Content-Type: text/plain
# repo = free-fedora-updates-released-9 arch = i386 country = US
http://mirror.liberty.edu/pub/rpmfusion/free/fedora/updates/9/i386
http://astromirror.uchicago.edu/rpmfusion/free/fedora/updates/9/i386
http://mirrors.cat.pdx.edu/rpmfusion/free/fedora/updates/9/i386
http://mirror.web-ster.com/rpmfusion/free/fedora/updates/9/i386
http://mirrors.tummy.com/mirrors/rpmfusion.org/free/fedora/updates/9/i386
-----Original Message-----
From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
Sent: Monday, April 27, 2009 7:54 PM
To: Dan Sopher
Cc: squid-users_at_squid-cache.org
Subject: Re: [squid-users] Proxy/no proxy GET results differ
> Hello.
>
> When running the following GET commands from a shell(same results from
> browsers), the results are different, with the error occurring when
> squid(Squid Cache: Version 3.0.STABLE13) is in the data path. Any help is
> appreciated. Thank you in advance.
You appear to be confused by proper HTTP behavior. The web server responds
in whatever way it needs to for the requests that comes in.
Squid is not capable of HTTP/1.1 and so sends HTTP/1.0 requests as it is
required to do.
The only possible thing like a bug I can see here is that squid _might_ be
transforming a TE: encoding request into a Content-Encoding: and the
client unable to handle such encodings despite needing to. We need the
headers input into squid from the client vs the headers sent to Server to
see.
Amos
>
>
> Examples and packet capture data is below:
>
>
> The following are the results WITHOUT squid in the data path:
>
> root_at_host ~# GET
> "http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386"
>
> # repo = free-fedora-updates-released-9 arch = i386 country = US
> http://astromirror.uchicago.edu/rpmfusion/free/fedora/updates/9/i386
> http://mirror.liberty.edu/pub/rpmfusion/free/fedora/updates/9/i386
> http://mirror.web-ster.com/rpmfusion/free/fedora/updates/9/i386
> http://mirrors.cat.pdx.edu/rpmfusion/free/fedora/updates/9/i386
> http://mirrors.tummy.com/mirrors/rpmfusion.org/free/fedora/updates/9/i386
>
>
>
> and, the following result is from using squid in the data path:
>
> root@host ~# GET -p http://192.168.0.22:3128
> "http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386"
> # Server Error
>
>
>
> PACKET CAPTURE DATA:
> ===================
> 1. This is the result without using Squid proxy. Traffic is captured from
> the host using this configuration:
>
> Data path: Host -> Juniper Netscreen firewall -> Router -> Internet
>
>
> GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1
> TE: deflate,gzip;q=0.3
> Connection: TE, close
> Host: mirrors.rpmfusion.org
> User-Agent: lwp-request/2.07
>
> HTTP/1.1 200 OK
> Date: Fri, 24 Apr 2009 20:53:22 GMT
> Server: Apache/2.2.11 (Fedora)
> Content-Length: 474
> cache-control: no-cache
> Connection: close
> Content-Type: text/plain
>
> # repo = free-fedora-updates-released-9 arch = i386 country = US
> http://astromirror.uchicago.edu/rpmfusion/free/fedora/updates/9/i386
> http://mirror.liberty.edu/pub/rpmfusion/free/fedora/updates/9/i386
> http://mirror.web-ster.com/rpmfusion/free/fedora/updates/9/i386
> http://mirrors.cat.pdx.edu/rpmfusion/free/fedora/updates/9/i386
> http://lordmorgul.net/pub/fedora/rpmfusion/free/fedora/updates/9/i386
> http://mirrors.tummy.com/mirrors/rpmfusion.org/free/fedora/updates/9/i386
>
>
>
>
> 2. This is the result using Squid proxy. Traffic captured from the
> external interface of the proxy:
>
> Data path: Host -> Squid proxy -> Juniper Netscreen firewall -> Router ->
> Internet
>
>
> GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.0
> Host: mirrors.rpmfusion.org
> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.9)
> Gecko/2009040821 Firefox/3.0.9 (.NET CLR 3.5.30729)
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Accept-Language: en-us,en;q=0.5
> Accept-Encoding: gzip,deflate
> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
> Keep-Alive: 300
> Via: 1.0 la-squid.twistbox.com (squid)
> X-Forwarded-For: unknown
> Cache-Control: max-age=259200
> Connection: keep-alive
>
> HTTP/1.1 200 OK
> Date: Fri, 24 Apr 2009 20:54:51 GMT
> Server: Apache/2.2.9 (Fedora)
> Content-Length: 35
> content-encoding: gzip
> cache-control: no-cache
> Connection: close
> Content-Type: text/plain; charset=UTF-8
>
> .....'.I..SV.N-*K-Rp-*./....$}N....
>
>
>
> CONFIGURATION:
> =============
> Squid Cache: Version 3.0.STABLE13
>
> configure options: '--build=i386-redhat-linux-gnu'
> '--host=i386-redhat-linux-gn
> u' '--target=i386-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr'
> '--exec-
> prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc'
> '--in
> cludedir=/usr/include' '--libdir=/usr/lib' '--libexecdir=/usr/libexec'
> '--shared
> statedir=/usr/com' '--mandir=/usr/share/man' '--infodir=/usr/share/info'
> '--exec
> _prefix=/usr' '--bindir=/usr/sbin' '--libexecdir=/usr/lib/squid'
> '--localstatedi
> r=/var' '--datadir=/usr/share' '--sysconfdir=/etc/squid'
> '--disable-dependency-t
> racking' '--enable-arp-acl' '--enable-auth=basic,digest,ntlm'
> '--enable-basic-au
> th-helpers=LDAP,MSNT,NCSA,PAM,SMB,YP,getpwnam,multi-domain-NTLM,SASL'
> '--enable-
> cache-digests' '--enable-cachemgr-hostname=localhost'
> '--enable-delay-pools' '--
> enable-digest-auth-helpers=password' '--enable-epoll'
> '--enable-external-acl-hel
> pers=ip_user,ldap_group,unix_group,wbinfo_group' '--enable-icap-client'
> '--enabl
> e-ident-lookups' '--with-large-files' '--enable-linux-netfilter'
> '--enable-ntlm-
> auth-helpers=SMB,fakeauth' '--enable-referer-log'
> '--enable-removal-policies=hea
> p,lru' '--enable-snmp' '--enable-ssl'
> '--enable-storeio=aufs,diskd,null,ufs' '--
> enable-useragent-log' '--enable-wccpv2' '--with-aio'
> '--with-default-user=squid'
> '--with-filedescriptors=16384' '--with-dl' '--with-openssl=/usr/kerberos'
> '--wi
> th-pthreads' 'build_alias=i386-redhat-linux-gnu'
> 'host_alias=i386-redhat-linux-g
> nu' 'target_alias=i386-redhat-linux-gnu' 'CFLAGS=-fPIE -Os -g -pipe
> -fsigned-cha
> r -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
> -fstack-protector --pa
> ram=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic
> -fasynchronous-unwind-tabl
> es' 'LDFLAGS=-pie' 'CXXFLAGS=-fPIE -O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -f
> exceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386
> -mtune=g
> eneric -fasynchronous-unwind-tables' 'FFLAGS=-O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_S
> OURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32
> -march=i38
> 6 -mtune=generic -fasynchronous-unwind-tables'
>
>
>
>
>
>
Received on Tue Apr 28 2009 - 18:39:31 MDT
This archive was generated by hypermail 2.2.0 : Wed Apr 29 2009 - 12:00:02 MDT