Hi,
I have collected some info from squid and the
redirector program wrt the malformed urls problem. It
is a huge mail, kindly spare some time to read
through.
Making a premise, the redirector code comprises of
two modules - the client is the one that is started by
squid. It fwds the requests to the redirector server
that takes care of processing the request. It then
sends the response to the client which inturn fwds the
response to the squid.
The bug is that malformed urls are getting
generated. The squid log for debug_options ALL,1 61,9
33,5 is also pasted below :-
1)
Client :-
12304 Thu Feb 26 18:11:04 2004: Client - read from
stdin = http://www.dictionary.com/
10.10.10.106/bhadra - GET
12305 Thu Feb 26 18:11:04 2004: Client - wrote to
server = http://www.dictionary.com/
10.10.10.106/bhadra - GET
12306 Thu Feb 26 18:11:04 2004: Client - the value
read from server =
http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106
10.10.10.106 mani GET
Server :-
72060 Thu Feb 26 18:11:04 2004: Server - the value
read = http://www.dictionary.com/ 10.10.10.106/bhadra
- GET
72061
72062 Thu Feb 26 18:11:04 2004: In loadInBuff, buff
= http://www.dictionary.com/ 10.10.10.106/bhadra -
GET
72098 Thu Feb 26 18:11:04 2004: in_buff.url =
http://www.dictionary.com/
72100 Thu Feb 26 18:11:04 2004: Server - wrote to
client =
http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106
10.10.10.106 mani GET
Cache.log :-
2004/02/26 18:11:04| parseHttpRequest: Method is
'GET'
2004/02/26 18:11:04| parseHttpRequest: URI is
'http://www.dictionary.com/'
2004/02/26 18:11:04| clientSetKeepaliveFlag:
http_ver = 1.0
2004/02/26 18:11:04| clientSetKeepaliveFlag: method
= GET
2004/02/26 18:11:04| The request GET
http://www.dictionary.com/ is ALLOWED, because it
matched 'all'
2004/02/26 18:11:04| redirectStart:
'http://www.dictionary.com/'
2004/02/26 18:11:04| redirectHandleRead:
{http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106
10.10.10.106 mani GET}
2004/02/26 18:11:04| clientRedirectDone:
'http://www.dictionary.com/'
result=http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106
2)
Client :-
12307 Thu Feb 26 18:11:04 2004: Client - read from
stdin =
http://update.messenger.yahoo.com/msgrcli.html
10.10.10.109/sharavathi - GET
12308 Thu Feb 26 18:11:04 2004: Client - wrote to
server =
http://update.messenger.yahoo.com/msgrcli.html
10.10.10.109/sharavathi - GET
12309 Thu Feb 26 18:11:05 2004: Client - read from
stdin = ñ^RBñ^RBww.dictionary.com/css/console.css
10.10.10.106/bhadra - GET
12310 Thu Feb 26 18:11:05 2004: Client - wrote to
server =
ñ^RBñ^RBww.dictionary.com/css/console.css
10.10.10.106/bhadra - GET
12311 Thu Feb 26 18:11:05 2004: Client - the value
read from server =
12312 Thu Feb 26 18:11:05 2004: Client - the value
read from server =
http://localhost:8080/contentfilter/login1.jsp?url=(%c3%c2%b1^RB%c3%c2%b1^RBww.dictionary.com/css/console.css)&ip=10.10.10.106
10.10.10.106 mani GET
Server :-
72708 Thu Feb 26 18:11:04 2004: Server - the value
read = http://update.messenger.yahoo.com/msgrcli.html
10.10.10.109/sharavathi - GET
72710 Thu Feb 26 18:11:04 2004: In loadInBuff, buff
= http://update.messenger.yahoo.com/msgrcli.html
10.10.10.109/sharavathi - GET
72134 Thu Feb 26 18:11:04 2004: in_buff.url =
http://update.messenger.yahoo.com/msgrcli.html
72135 Thu Feb 26 18:11:04 2004: After doAuth,
in_buff.url = http://update.me
ssenger.yahoo.com/msgrcli.html
Thu Feb 26 18:11:05 2004: Allowed , wrote to client
=
Cache.log :-
2004/02/26 18:11:04| parseHttpRequest: Method is
'GET'
2004/02/26 18:11:04| parseHttpRequest: URI is
'http://update.messenger.yahoo.com/msgrcli.html'
2004/02/26 18:11:04| clientSetKeepaliveFlag: http_ver
= 1.0
2004/02/26 18:11:04| clientSetKeepaliveFlag: method =
GET
2004/02/26 18:11:04| The request GET
http://update.messenger.yahoo.com/msgrcli.html is
ALLOWED, because it matched 'all'
2004/02/26 18:11:04| The request GET
http://update.messenger.yahoo.com/msgrcli.html is
ALLOWED, because it matched 'all'
2004/02/26 18:11:04| redirectStart:
'http://update.messenger.yahoo.com/msgrcli.html'
2004/02/26 18:11:04| clientSendMoreData:
http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106,
3881 bytes
2004/02/26 18:11:04| clientSendMoreData: FD 16
'http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106',
out.offset=0
2004/02/26 18:11:04| clientBuildReplyHeader: can't
keep-alive, unknown body size
2004/02/26 18:11:04| clientSendMoreData: Appending
3584 bytes after 297 bytes of headers
2004/02/26 18:11:04| The reply for GET
http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106
is ALLOWED, because it matched 'all'
2004/02/26 18:11:05| clientWriteComplete: FD 16, sz
3914, err 0, off 3881, len -1
2004/02/26 18:11:05| clientSendMoreData:
http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106,
1349 bytes
2004/02/26 18:11:05| clientSendMoreData: FD 16
'http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106',
out.offset=3881
2004/02/26 18:11:05| clientWriteComplete: FD 16, sz
1349, err 0, off 5230, len -1
2004/02/26 18:11:05| clientSendMoreData:
http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106,
0 bytes
2004/02/26 18:11:05| clientSendMoreData: FD 16
'http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106',
out.offset=5230
2004/02/26 18:11:05| clientWriteComplete: FD 16, sz
0, err 0, off 5230, len 5230
2004/02/26 18:11:05| clientWriteComplete: FD 16
transfer is DONE
2004/02/26 18:11:05| clientWriteComplete: closing,
content_length < 0
2004/02/26 18:11:05| connStateFree: FD 16
2004/02/26 18:11:05| httpRequestFree:
http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/)&ip=10.10.10.106
2004/02/26 18:11:05| httpAccept: FD 16: accepted
2004/02/26 18:11:05| clientReadRequest: FD 16:
reading request...
2004/02/26 18:11:05| parseHttpRequest: Method is
'GET'
2004/02/26 18:11:05| parseHttpRequest: URI is
'http://www.dictionary.com/css/console.css'
3)
Client :-
12307 Thu Feb 26 18:11:04 2004: Client - read from
stdin =
http://update.messenger.yahoo.com/msgrcli.html
10.10.10.109/sharavathi - GET
12308 Thu Feb 26 18:11:04 2004: Client - wrote to
server =
http://update.messenger.yahoo.com/msgrcli.html
10.10.10.109/sharavathi - GET
12309 Thu Feb 26 18:11:05 2004: Client - read from
stdin = ñ^RBñ^RBww.dictionary.com/css/console.css
10.10.10.106/bhadra - GET
12310 Thu Feb 26 18:11:05 2004: Client - wrote to
server =
ñ^RBñ^RBww.dictionary.com/css/console.css
10.10.10.106/bhadra - GET
12311 Thu Feb 26 18:11:05 2004: Client - the value
read from server =
12312 Thu Feb 26 18:11:05 2004: Client - the value
read from server =
http://localhost:8080/contentfilter/login1.jsp?url=(%c3%c2%b1^RB%c3%c2%b1^RBww.dictionary.com/css/console.css)&ip=10.10.10.106
10.10.10.106 mani GET
Server :-
72181 Thu Feb 26 18:11:05 2004: Server - the value
read = ñ^RBñ^RBww.dictionary.com/css/console.css
10.10.10.106/bhadra - GET
72182
72183 Thu Feb 26 18:11:05 2004: In loadInBuff, buff
= ñ^RBñ^RBww.dictionary.com/css/console.css
10.10.10.106/bhadra - GET
72219 Thu Feb 26 18:11:05 2004: in_buff.url =
ñ^RBñ^RBww.dictionary.com/css/console.css
72220 Thu Feb 26 18:11:05 2004: Going to send to as
identity = mani
72221 Thu Feb 26 18:11:05 2004: Server - wrote to
client =
http://localhost:8080/contentfilter/login1.jsp?url=(%c3%c2%b1^RB%c3%c2%b1^RBww.dictionary.com/css/console.css)&ip=10.10.10.106
10.10.10.106 mani GET
Cache.log :-
2004/02/26 18:11:05| parseHttpRequest: Method is
'GET'
2004/02/26 18:11:05| parseHttpRequest: URI is
'http://www.dictionary.com/css/console.css'
2004/02/26 18:11:05| clientSetKeepaliveFlag:
http_ver = 1.0
2004/02/26 18:11:05| clientSetKeepaliveFlag: method
= GET
2004/02/26 18:11:05| The request GET
http://www.dictionary.com/css/console.css is ALLOWED,
because it matched 'all'
2004/02/26 18:11:05| redirectStart:
'http://www.dictionary.com/css/console.css'
2004/02/26 18:11:05| redirectHandleRead: {}
2004/02/26 18:11:05| clientRedirectDone:
'http://www.dictionary.com/css/console.css'
result=NULL
4)
Client :-
12313 Thu Feb 26 18:11:06 2004: Client - read from
stdin =
http://dictionary.reference.com/css/console.css
10.10.10.106/bhadra - GET
12314 Thu Feb 26 18:11:06 2004: Client - wrote to
server =
http://dictionary.reference.com/css/console.css
10.10.10.106/bhadra - GET
12315 Thu Feb 26 18:11:06 2004: Client - the value
read from server =
http://localhost:8080/contentfilter/login1.jsp?url=(http://dictionary.reference.com/css/console.css)&ip=10.10.10.106
10.10.10.106 mani GET
Server :-
72225 Thu Feb 26 18:11:06 2004: Server - the value
read =
http://dictionary.reference.com/css/console.css
10.10.10.106/bhadra - GET
72226
72227 Thu Feb 26 18:11:06 2004: In loadInBuff, buff
= http://dictionary.reference.com/css/console.css
10.10.10.106/bhadra - GET
72263 Thu Feb 26 18:11:06 2004: in_buff.url =
http://dictionary.reference.com/css/console.css
72264 Thu Feb 26 18:11:06 2004: Going to send to as
identity = mani
72265 Thu Feb 26 18:11:06 2004: Server - wrote to
client =
http://localhost:8080/contentfilter/login1.jsp?url=(http://dictionary.reference.com/css/console.css)&ip=10.10.10.106
10.10.10.106 mani GET
Cache.log :-
2004/02/26 18:11:06| parseHttpRequest: Method is
'GET'
2004/02/26 18:11:06| parseHttpRequest: URI is
'http://dictionary.reference.com/css/console.css'
2004/02/26 18:11:06| clientSetKeepaliveFlag:
http_ver = 1.0
2004/02/26 18:11:06| clientSetKeepaliveFlag: method
= GET
2004/02/26 18:11:06| The request GET
http://dictionary.reference.com/css/console.css is
ALLOWED, because it matched 'all'
2004/02/26 18:11:06| redirectStart:
'http://dictionary.reference.com/css/console.css'
2004/02/26 18:11:06| redirectHandleRead:
{http://localhost:8080/contentfilter/login1.jsp?url=(http://dictionary.reference.com/css/console.css)&ip=10.10.10.106
10.10.10.106 mani GET}
2004/02/26 18:11:06| clientRedirectDone:
'http://dictionary.reference.com/css/console.css'
result=http://localhost:8080/contentfilter/login1.jsp?url=(http://dictionary.reference.com/css/console.css)&ip=10.10.10.106
5)
Client :-
12316 Thu Feb 26 18:11:06 2004: Client - read from
stdin = http://www.dictionary.com/images/1x1.gif
10.10.10.106/bhadra - GET
12317 Thu Feb 26 18:11:06 2004: Client - wrote to
server = http://www.dictionary.com/images/1x1.gif
10.10.10.106/bhadra - GET
12318 Thu Feb 26 18:11:06 2004: Client - the value
read from server =
http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/images/1x1.gif)&ip=10.10.10.106
10.10.10.106 mani GET
Server :-
72269 Thu Feb 26 18:11:06 2004: Server - the value
read = http://www.dictionary.com/images/1x1.gif
10.10.10.106/bhadra - GET
72270
72271 Thu Feb 26 18:11:06 2004: In loadInBuff, buff
= http://www.dictionary.com/images/1x1.gif
10.10.10.106/bhadra - GET
72307 Thu Feb 26 18:11:06 2004: in_buff.url =
http://www.dictionary.com/images/1x1.gif
72308 Thu Feb 26 18:11:06 2004: Going to send to as
identity = mani
72309 Thu Feb 26 18:11:06 2004: Server - wrote to
client =
http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/images/1x1.gif)&ip=10.10.10.106
10.10.10.106 mani GET
Cache.log :-
2004/02/26 18:11:06| parseHttpRequest: Method is
'GET'
2004/02/26 18:11:06| parseHttpRequest: URI is
'http://www.dictionary.com/images/1x1.gif'
2004/02/26 18:11:06| clientSetKeepaliveFlag: http_ver
= 1.0
2004/02/26 18:11:06| clientSetKeepaliveFlag: method =
GET
2004/02/26 18:11:06| The request GET
http://www.dictionary.com/images/1x1.gif is ALLOWED,
because it matched 'all'
2004/02/26 18:11:06| redirectStart:
'http://www.dictionary.com/images/1x1.gif'
2004/02/26 18:11:06| redirectHandleRead:
{http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/images/1x1.gif)&ip=10.10.10.106
10.10.10.106 mani GET}
2004/02/26 18:11:06| clientRedirectDone:
'http://www.dictionary.com/images/1x1.gif'
result=http://localhost:8080/contentfilter/login1.jsp?url=(http://www.dictionary.com/images/1x1.gif)&ip=10.10.10.106
6)
Client :-
12327 Thu Feb 26 18:11:11 2004: Client - read from
stdin =
ñ^RBñ^RBocalhost:8080/contentfilter/login2.jsp
10.10.10.106/bhadra - POST
12328 Thu Feb 26 18:11:11 2004: Client - wrote to
server =
ñ^RBñ^RBocalhost:8080/contentfilter/login2.jsp
10.10.10.106/bhadra - POST
12330 Thu Feb 26 18:11:11 2004: Client - the value
read from server =
http://localhost:8080/contentfilter/login1.jsp?url=(%c3%c2%b1^RB%c3%c2%b1^RBocalhost:8080/contentfilter/login2.jsp)&ip=10.10.10.106
10.10.10.106 mani POST
Server :-
72519 Thu Feb 26 18:11:11 2004: Server - the value
read =
ñ^RBñ^RBocalhost:8080/contentfilter/login2.jsp
10.10.10.106/bhadra - POST
72520
72521 Thu Feb 26 18:11:11 2004: In loadInBuff,
buff =
ñ^RBñ^RBocalhost:8080/contentfilter/login2.jsp
10.10.10.106/bhadra - POST
72557 Thu Feb 26 18:11:11 2004: in_buff.url =
ñ^RBñ^RBocalhost:8080/contentfilter/login2.jsp
72558 Thu Feb 26 18:11:11 2004: Going to send to
as identity = mani
72559 Thu Feb 26 18:11:11 2004: Server - wrote to
client =
http://localhost:8080/contentfilter/login1.jsp?url=(%c3%c2%b1^RB%c3%c2%b1^RBocalhost:8080/contentfilter/login2.jsp)&ip=10.10.10.106
10.10.10.106 mani POST
Cache.log :-
2004/02/26 18:11:11| parseHttpRequest: Method is
'POST'
2004/02/26 18:11:11| parseHttpRequest: URI is
'http://localhost:8080/contentfilter/login2.jsp'
2004/02/26 18:11:11| clientSetKeepaliveFlag: http_ver
= 1.0
2004/02/26 18:11:11| clientSetKeepaliveFlag: method =
POST
2004/02/26 18:11:11| The request POST
http://localhost:8080/contentfilter/login2.jsp is
ALLOWED, because it matched 'all'
2004/02/26 18:11:11| redirectStart:
'http://localhost:8080/contentfilter/login2.jsp'
2004/02/26 18:11:11| redirectHandleRead:
{http://localhost:8080/contentfilter/login1.jsp?url=(%dc%b1B%dc%b1Bww.dictionary.com/css/console.css)&ip=10.10.10.106
10.10.10.106 mani GET}
2004/02/26 18:11:11| clientRedirectDone:
'http://localhost:8080/contentfilter/login2.jsp'
result=http://localhost:8080/contentfilter/login1.jsp?url=(%dc%b1B%dc%b1Bww.dictionary.com/css/console.css)&ip=10.10.10.106
7)
Client :-
12331 Thu Feb 26 18:11:11 2004: Client - read from
stdin =
http://localhost:8080/contentfilter/css/console.css
10.10.10.106/bhadra - GET
12332 Thu Feb 26 18:11:11 2004: Client - wrote to
server =
http://localhost:8080/contentfilter/css/console.css
10.10.10.106/bhadra - GET
12333 Thu Feb 26 18:11:11 2004: Client - the value
read from server =
Server :-
72563 Thu Feb 26 18:11:11 2004: Server - the value
read =
http://localhost:8080/contentfilter/css/console.css
10.10.10.106/bhadra - GET
72564
72565 Thu Feb 26 18:11:11 2004: In loadInBuff,
buff =
http://localhost:8080/contentfilter/css/console.css
10.10.10.106/bhadra - GET
72568 Thu Feb 26 18:11:11 2004: The request is
for url =
http://localhost:8080/contentfilter/login2.jsp
72569 Thu Feb 26 18:11:11 2004: Server - wrote to
client =
Cache.log :-
2004/02/26 18:11:11| parseHttpRequest: Method is
'GET'
2004/02/26 18:11:11| parseHttpRequest: URI is
'http://localhost:8080/contentfilter/css/console.css'
2004/02/26 18:11:11| clientSetKeepaliveFlag:
http_ver = 1.0
2004/02/26 18:11:11| clientSetKeepaliveFlag: method
= GET
2004/02/26 18:11:11| The request GET
http://localhost:8080/contentfilter/css/console.css is
ALLOWED, because it matched 'all'
2004/02/26 18:11:11| redirectStart:
'http://localhost:8080/contentfilter/css/console.css'
2004/02/26 18:11:11| clientSendMoreData:
http://localhost:8080/contentfilter/login1.jsp?url=(%dc%b1B%dc%b1Bww.dictionary.com/css/console.css)&ip=10.10.10.106,
0 bytes
2004/02/26 18:11:11| clientSendMoreData: FD 18
'http://localhost:8080/contentfilter/login1.jsp?url=(%dc%b1B%dc%b1Bww.dictionary.com/css/console.css)&ip=10.10.10.106',
out.offset=5230
8)
Client :-
12334 Thu Feb 26 18:11:11 2004: Client - read from
stdin =
http://localhost:8080/contentfilter/images/1x1.gif
10.10.10.106/bhadra - GET
12335 Thu Feb 26 18:11:11 2004: Client - wrote to
server =
http://localhost:8080/contentfilter/images/1x1.gif
10.10.10.106/bhadra - GET
12336 Thu Feb 26 18:11:11 2004: Client - the
value read from server =
Server :-
72573 Thu Feb 26 18:11:11 2004: Server - the value
read =
http://localhost:8080/contentfilter/images/1x1.gif
10.10.10.106/bhadra - GET
72574
72575 Thu Feb 26 18:11:11 2004: In loadInBuff,
buff =
http://localhost:8080/contentfilter/images/1x1.gif
10.10.10.106/bhadra - GET
72578 Thu Feb 26 18:11:11 2004: The request is
for url =
http://localhost:8080/contentfilter/login2.jsp
72579 Thu Feb 26 18:11:11 2004: Server - wrote to
client =
Cache.log :-
2004/02/26 18:11:11| parseHttpRequest: Method is
'GET'
2004/02/26 18:11:11| parseHttpRequest: URI is
'http://localhost:8080/contentfilter/images/1x1.gif'
2004/02/26 18:11:11| clientSetKeepaliveFlag:
http_ver = 1.0
2004/02/26 18:11:11| clientSetKeepaliveFlag: method
= GET
2004/02/26 18:11:11| The request GET
http://localhost:8080/contentfilter/images/1x1.gif is
ALLOWED, because it matched 'all'
2004/02/26 18:11:11| redirectStart:
'http://localhost:8080/contentfilter/images/1x1.gif'
2004/02/26 18:11:11| redirectHandleRead: {}
2004/02/26 18:11:11| clientRedirectDone:
'http://localhost:8080/contentfilter/images/1x1.gif'
result=NULL
My observations :-
1) In the second case(2)), squid does a redirectStart
but doesn't wait to receive the response that is there
is no redirectHandleRead message received but the
redirector program is sending back a response.
2) The response from the second redirectStart is read
as response for the third redirectStart and hence the
confusion starts and all the subsequent messages get
corrupted.
Plz tell me why this is so and a solution if
possible. Thanks for ur time once again.
Regards and TIA,
Deepa
________________________________________________________________________
Yahoo! India Insurance Special: Be informed on the best policies, services, tools and more.
Go to: http://in.insurance.yahoo.com/licspecial/index.html
Received on Mon Mar 01 2004 - 03:10:53 MST
This archive was generated by hypermail pre-2.1.9 : Thu Apr 01 2004 - 12:00:01 MST
