[squid-users] Re: wget and ftp

Seems to be a bug in wget.

822
823 if (opt.recursive && url_scheme (*t) != SCHEME_FTP)
824 status = retrieve_tree (*t);
825 else
826 status = retrieve_url (*t, &filename, &redirected_URL,
NULL, &dt);
827

tells that if you are requesting a FTP object then it won't do a
recursive retrieval. The fact that it does a recursive retrieval when
not using a proxy is only because they also reimplemented retrieve_tree
within low level ftp protocol implementation which is obviously not used
when using a proxy...

Change the if to

  if (opt.recursive && (url_scheme (*t) != SCHEME_FTP || opt.use_proxy))

and it should work a bit better if you specify the -Y command line
option.. (or --proxy)

Doing a proper fix requires a fair bit more of coding in wget..
(basically the reason why the SCHEME_FTP condition is there in the first
place should be eliminated, whatever that reason may be)

Regards
Henrik

ons 2003-04-09 klockan 22.37 skrev Sampei:
> Can yuo help me about this problem please ?
>
> I'm using proxy Squid Stable 2.5 in my company
> and I want to download all binary files from a "public ftp server"
> into my internal server.
>
> Proxy squid: RedHat 7.3
>
> I created a script and I run it into my internal server.
> In this script I set "ftp_proxy variable" with its address ip and its port,
> and I exported this variable.
>
> WGET ftp_proxy... -m ... ftp://...directory/
>
> Squid gives to wget an INDEX.HTML file,
> which contains a files list of the directory
> specified in ftp://...directory/ but in HTML mode,
> so it's impossible for wget to translate file names correctly.
> Infact wget result fails.
>
> I thought filtering index.html into a clean file (e.g. called "clean") and
> using wget to download binary files once to once,
> which are indicated into this "clean" file.
>
> Problem reappears every time that I have a SUB-directory,
> infact index.html file is recreated .
>
> Do you know the way to solve this problem,
> to download whole directory structure (files included) into my ftp server
> using squid ?
>
> thanks
>
> Riccardo

-- 
Free Squid-users support provided by Henrik Nordström <hno@squid-cache.org>
Donations welcome if you consider my Free Squid support helpful.
https://www.paypal.com/xclick/business=hno%40squid-cache.org
If you need commercial Squid support or cost effective Squid and
firewall appliances please refer to MARA Systems AB, Sweden
http://www.marasystems.com/, info@marasystems.com