Re: [squid-users] invalid url in curl

From: Amos Jeffries <squid3_at_treenet.co.nz>
Date: Fri, 12 Aug 2011 05:25:19 +1200

On 11/08/11 07:56, Mohsen Pahlevanzadeh wrote:
> On Wed, 2011-08-10 at 16:56 +1200, Amos Jeffries wrote:
>> On Tue, 09 Aug 2011 19:57:47 +0430, Mohsen Pahlevanzadeh wrote:
>>> On Wed, 2011-08-10 at 01:20 +1200, Amos Jeffries wrote:
>>>> On 10/08/11 00:31, Mohsen Pahlevanzadeh wrote:
>>>>> Dear all,
>>>>>
>>>>> I recall when use squidclient "url" and our url is not valid, now
>>>> i
>>>>> tested with telnet google, and save html file too google.html and
>>>>> changed a bit its header for cacheable:
>>>>> --------------
>>>>> curl -H "HTTP/1.1 200 OK" -H "Date: Tue, 09 Aug 2011 12:12:54 GMT"
>>>> -H
>>>>> "Expires: Thu, 08 Sep 2011 12:12:54 GMT" -H "Cache-Control:
>>>> public,
>>>>> max-age=29000" -H "Location: http://www.google.com/" -H
>>>> "Content-Type:
>>>>> text/html; charset=ISO-8859-1" -H "Server: gws" -H
>>>> "X-XSS-Protection:
>>>>> 1; mode=block" -H "X-Cache: MISS from debian" -H
>>>> "Transfer-Encoding:
>>>>> chunked" -d @./files/Google.com/Google.html localhost:3128
>>>>> ---------------
>>>>> But i receive the bad url err.i don't know how to put url.url does
>>>>> initialize with "Location" header ? if yes squid doesn't get
>>>> error.
>>>>>
>>>>> again i'm reading rfc, but i think it associate to acceptance of
>>>> squid.
>>>>> How i hand off file with curl to squid?
>>>>
>>>> curl is a client software. Just like a browser. It _receives_ files
>>>> from
>>>> squid. It does not send. Only web servers and proxies send page
>>>> objects
>>>> in HTTP.
>>>>
>>>> curl --proxy 127.0.0.1:3128 http://www.google.com/
>>>>
>>>> Request sent to squid:
>>>> ---------------
>>>> GET http://www.google.com/ HTTP/1.0
>>>> Host: www.google.com
>>>> User-Agent: curl
>>>> Accept: */*
>>>> Proxy-Authorization: Basic ***==
>>>> Connection: close
>>>>
>>>> ---------------
>>>>
>>>> squid at 127.0.0.1 contacts www.google.com,
>>>> www.google.com sends the Reply to squid.
>>>> squid sends it to curl
>>>>
>>>> Reply that comes back to curl:
>>>> ---------------
>>>> HTTP/1.1 302 Moved Temporarily
>>>> Location: http://www.google.co.nz/
>>>> Cache-Control: private
>>>> Content-Type: text/html; charset=UTF-8
>>>> Set-Cookie: PREF=ID=***:FF=0:TM=1312894424:LM=1312894424:S=***;
>>>> expires=Thu, 08-Aug-2013 12:53:44 GMT; path=/; domain=.google.com
>>>> Date: Tue, 09 Aug 2011 12:53:44 GMT
>>>> Server: gws
>>>> Content-Length: 221
>>>> X-XSS-Protection: 1; mode=block
>>>> X-Cache: MISS from treenet.co.nz
>>>> X-Cache-Lookup: MISS from treenet.co.nz:8080
>>>> Via: 1.1 treenet.co.nz (squid/3.3.0.0)
>>>> Connection: close
>>>>
>>>> ---------------
>>>>
>>>> A normal web browser would follow that 302 redirect instruction
>>>> and try again with second Request to squid ....
>>>>
>>>> Request sent to squid:
>>>> ---------------
>>>> GET http://www.google.co.nz/ HTTP/1.0
>>>> Host: www.google.co.nz
>>>> User-Agent: curl
>>>> Accept: */*
>>>> Proxy-Authorization: Basic ***==
>>>> Connection: close
>>>>
>>>> ---------------
>>>>
>>>> squid at 127.0.0.1 contacts www.google.co.nz,
>>>> www.google.co.nz sends the Reply to squid.
>>>> squid sends it to curl:
>>>>
>>>> Reply that comes back to curl:
>>>> ---------------
>>>> HTTP/1.1 200 OK
>>>> Date: Tue, 09 Aug 2011 13:01:27 GMT
>>>> Expires: -1
>>>> Cache-Control: private, max-age=0
>>>> Content-Type: text/html; charset=ISO-8859-1
>>>> Set-Cookie: PREF=ID=***:FF=0:TM=1312894887:LM=1312894887:S=***;
>>>> expires=Thu, 08-Aug-2013 13:01:27 GMT; path=/; domain
>>>> =.google.co.nz
>>>> Set-Cookie: NID=49=***; expires=Wed, 08-Feb-2012 13:01:27 GMT;
>>>> path=/;
>>>> domain=.google.co.nz; HttpOnly
>>>> Server: gws
>>>> X-XSS-Protection: 1; mode=block
>>>> X-Cache: MISS from treenet.co.nz
>>>> X-Cache-Lookup: MISS from treenet.co.nz:8080
>>>> Via: 1.1 treenet.co.nz (squid/3.3.0.0)
>>>> Connection: close
>>>>
>>>> <!doctype html><html><head><meta http-equiv="content-type"
>>>> content="text/html;
>>>> charset=ISO-8859-1"><title>Google</title><script>window.google={
>>>> ---------------
>>>>
>>>> Amos
>>> If i send Obejct files in the directory, can i get result? how can i
>>> simulate?
>>> --mohsen
>>
>> The above was a simulation I created by actually doing the command:
>> curl --proxy 127.0.0.1:3128 http://www.google.com/
>>
>>
>> The object files are in a directory '/' on the machine www.google.co.nz
>> accessed through a http:// service.
>>
>> If you have an file transfer service (FTP) on the local machine a file
>> called "name" might be available to Squid as the URL
>> "ftp://localhost/name"
>>
>> Amos
>>
> You say squid-box accept from origin server or proxy server, not browser
> or any client machine,So it's very interesting for me that i know its
> reason.squid can see in files that i can't send by hand?(sequence of
> objects,headers and etc.)
> --mohsen

reason? because that is the way it was designed to be.

  "requests shall come from clients when HTTP is happening"
  "replies shall come from servers when HTTP is happening"

no less fundamental principles than:
  "rain shall come from the sky when gravity is happening"

Other designs have other names. This is HTTP design.

You can be a server "by hand". Open a socket and listen on it until
somebody (squid) connects and hands you a request for data. You then
send the reply for that requested URL.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.14
   Beta testers wanted for 3.2.0.10
Received on Thu Aug 11 2011 - 17:25:27 MDT

This archive was generated by hypermail 2.2.0 : Fri Aug 12 2011 - 12:00:01 MDT