Re: Question...

From: Lionel Bouton <Lionel.Bouton@dont-contact.us>
Date: Mon, 11 May 1998 14:12:30 +0200

Guillaume LACHENAL wrote:

> Hi,
>
> Where can i found a solution to automaticly folow link of a web site
> to put all files of this site into my proxy ?
>
> Thx !

Just an idea, not a complete solution, this would need some coding
effort :
You could use a redirector which would call a prefecth engine using wget
to download every (\.html$|\.htm$) files then an HTML parser for links
extraction. Then the prefecth engine would reuse wget to prefetch these
links, but it will have to tell the redirector NOT to prefetch links
from prefetched pages before asking them to avoid prefetch loops and a
complete site download (or a complete web download if you prefetch links
from sites to sites).
Prefetching is not always a good solution:
you win on latency but you waste bandwith.
If a standard way existed for bandwith use determination, prefetching
could be far clever.

Lionel Bouton.
Received on Mon May 11 1998 - 05:18:06 MDT

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:40:07 MST