Re: [squid-users] Reverse Proxy, multiple web servers, only one is reachable

From: Joaquín Puga <jpdelrio_at_gmail.com>
Date: Wed, 20 May 2009 15:12:23 -0700

On Tue, May 19, 2009 at 7:22 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
>> Hi there.
>>
>> Currently we are running squid 2.5.STABLE3 under RHEL3. However, this
>> week our ssl certificate will expire and the new certificate is a
>> chained certificate, which is not supported by that version of squid.
>> Also it is an old server in need of an upgrade, so we are trying to
>> configure squid 2.6.STABLE21 (running under RHEL 5.3) as a reverse
>> proxy, but after reading the documentation, the FAQ and many emails
>> from the email lists we still can't figure out what we are doing
>> wrong.
>>
>> - We have 4 web sites with public IPs x.y.z.47, x.y.z.48, x.y.z.49 and
>> x.y.z.50.
>> Each web site is hosted on a different server with Ips x.y.z.247,
>> x.y.z.248, x.y.z.249 and x.y.z.250 (x.y.z.47 goes to x.y.z.247, etc)
>> Our DNS server runs on the same box as squid.
>>
>> - x.y.z.48 is using ssl connections.
>>
>> - With the current configuration www.mywebsite.ca and
>> www1.mywebsite.ca work, but when trying to go to the other websites we
>> get to www.mywebsite.ca instead.
>>
>> If we remove the # from the cache_peer_domain lines then the only
>> website accessible is www1.mywebsite.ca. The other websites time out
>> and we get this error message:
>>
>> ERROR
>> The requested URL could not be retrieved
>>
>> While trying to retrieve the URL: http://www.mywebsite.ca/
>>
>> The following error was encountered:
>>
>>     * Unable to forward this request at this time.
>>
>> This request could not be forwarded to the origin server or to any
>> parent caches. The most likely cause for this error is that:
>>
>>     * The cache administrator does not allow this cache to make direct
>> connections to origin servers, and
>>     * All configured parent caches are currently unreachable.
>>
>> Your cache administrator is root.
>> Generated Tue, 19 May 2009 17:16:35 GMT by www1.mywebsite.ca
>> (squid/2.6.STABLE21)
>>
>> - It's our understanding that squid uses /etc/squid/hosts to have the
>> hostnames redefined and to get traffic to the backend servers. So if
>> the client requests www.mywebsite.ca, with dns record is x.y.z.47,
>> squid uses the hosts file to resolve www.mywebsite to x.y.z.247. Is
>> this correct?
>
> Not for reverse proxies. The destination is solely dependant on the
> 'address/host' value in cache_peer. If its an IP that is used. If its a
> FQDN then DNS is checked on startup/reconfigure. Hosts file overrides DNS.
>
> Your attempted squid.conf using IPs (x.y.z.247 etc) is the best way to go.
>
>>
>> - We also want to avoid people connecting to the websites using any
>> Ips (either x.y.z.47, .48, etc or x.y.z.247, .248, etc)
>>
>
> see notes inline with your 2.6 config.
>
>>
>> Below you can find the configuration files. Please let me know if you
>> need more information. I'd really appreciate if you could point me in
>> the right direction.
>>
>> #Squid.conf [version 2.5.STABLE3]:
>> #-----------------------------------------------------
>> http_port 80
>> https_port x.y.z.48:443 cert=/etc/squid/certs/ww1.pem
>> key=/etc/squid/certs/ww1key.pem version=1
>> icp_port 0
>> cache_dir null /tmp
>> acl all_no_cache src 0/0
>> no_cache deny all_no_cache
>> #Path to the host file hosts_file /etc/squid/hosts
>> httpd_accel_host virtual
>> httpd_accel_uses_host_header on
>> visible_hostname www1.mywebsite.ca
>> acl all src 0.0.0.0/0.0.0.0
>> acl mynet src x.y.z.0/255.255.255.0
>> http_access allow all
>> http_access allow mynet
>> http_access deny all
>>
>>
>> #squid.conf version 2.6.STABLE21
>> #-------------------------------------------------
>> acl all src 0.0.0.0/0.0.0.0
>> acl manager proto cache_object
>> acl localhost src 127.0.0.1/255.255.255.255
>> acl to_localhost dst 127.0.0.0/8
>> acl SSL_ports port 443
>> acl CONNECT method CONNECT
>> acl mynet src x.y.z.0/255.255.255.0
>
>> http_access allow all
>> http_access allow mynet
>> http_access allow localhost
>> http_access deny all
>> icp_access allow all
>
> Kill all of the above http_access and icp_access. It's not needed and
> prevents Squid from halting bad requests early in the process.
>
>>
>> http_port 80 accel vhost
>> https_port x.y.z.48:443 cert=/etc/squid/certs/ww1.pem
>> key=/etc/squid/certs/ww1key.pem version=1 accel vhost
>
> Correct.
>
>>
>> cache_peer x.y.z.247 parent 80 0 no-query no-digest originserver
>> name=www_mywebsite
>> cache_peer x.y.z.248 parent 80 0 no-query no-digest originserver
>> name=www1_mywebsite
>> cache_peer x.y.z.249 parent 80 0 no-query no-digest originserver
>> name=www_mywebsiteusa
>> cache_peer x.y.z.250 parent 80 0 no-query no-digest originserver
>> name=webmail
>
> Correct.
>
> Here is where things go askew slightly. You need some controls to branch
> the requests to the right peer based on the domain wanted.
>
>>
>> #cache_peer_domain www_mywebsite www.mywebsite.ca
>> #cache_peer_domain www1_mywebsite www1.mywebsite.ca
>> #cache_peer_domain www_mywebsiteusa www.mywebsiteusa.com
>> #cache_peer_domain webmail web.mywebsite.ca
>
> They should work. It's the crude hammer way to do it, but simple when you
> don't have sub-domain clauses (ie *.mywebsite.ca EXCEPT www1.mywebsite.ca
> and webmail.mywebsite.ca).
>
> If you only want www.mywebsite.ca, www1.mywebsite.ca, webmail.mywebsite.ca
> then these will work for you.
>
> Finer control can be gained using ACLs (as you defined below):
>
>>
>> #acl acl_www_mywebsite dstdomain www.mywebsite.ca
>> #acl acl_www1_mywebsite dstdomain www1.mywebsite.ca
>> #acl acl_www_mywebsiteusa dstdomain www.mywebsiteusa.com
>> #acl acl_webmail dstdomain webmail.mywebsite.ca
>
> But these only define the ACL lists, they still need to be used:
>
>  cache_peer_access www_mywebsite allow acl_www_mywebsite
>  cache_peer_access www_mywebsite deny all
>  http_access allow acl_www_mywebsite
>
> ... repeat for each peer.
>
> ... finish with:
>  http_access deny all
>
>>
>> cache_dir null /tmp
>
> Make sure you have as much RAM as possible and set cache_mem to as big a
> value as you can without causing swapping under peak loads.
>
> <snip remainder>
>>
>> #/etc/squid/hosts
>> -----------------------------------------------------------
>> x.y.z.247     www.mywebsite.ca
>> x.y.z.248     www1.mywebsite.ca
>> x.y.z.249     www.mywebsiteusa.com
>> x.y.x.250     webmail.mywebsite.ca
>>
>> Thanks a lot.
>>
>> Joaquin Puga.
>>
>
> HTH
> Amos
>
>
>

Thanks Amos and Chris for getting back to me so quickly. Unfortunately
none of my test ended up working. I have tried to use
cache_peer_domain and cache_peer_access + acls, and different
combinations but none of them had worked out. Nevertheless, no matter
the configuration www1.mywebsite.ca is always accessible, and I can't
understand why. Also, with my initial configuration,
www.mywebsiteusa.com is not displayed, but it shows www.mywebsite.ca.
And I cannot understand this either.

Below you can find a summation of most the configuration tests I tried
today based on your recommendations.

Our configuration should be very simple and straightforward, but it's
giving me a really hard time...
Any thoughts on what could be wrong?

[1] Amos
----------------------------------------------------------------
cache_peer x.y.z.247 parent 80 0 no-query no-digest originserver
name=www_mywebsite
cache_peer x.y.z.248 parent 80 0 no-query no-digest originserver
name=www1_mywebsite
cache_peer x.y.z.249 parent 80 0 no-query no-digest originserver
name=www_mywebsiteusa
cache_peer x.y.z.250 parent 80 0 no-query no-digest originserver name=webmail

cache_peer_domain www_mywebsite www.mywebsite.ca
cache_peer_domain www1_mywebsite www1.mywebsite.ca
cache_peer_domain www_mywebsiteusa www.mywebsiteusa.com
cache_peer_domain webmail webmail.mywebsite.ca

result [1]:
--------------------------
ERROR
The requested URL could not be retrieved

While trying to retrieve the URL: http://www1.mywebsite.ca/

The following error was encountered:

* Access Denied.

Access control configuration prevents your request from being allowed
at this time. Please contact your service provider if you feel this is
incorrect.

Your cache administrator is root.
Generated Wed, 20 May 2009 17:55:02 GMT by www1.mywebsite.ca
(squid/2.6.STABLE21)

[1.1]
-----------------------------------------------
re-enabling the following lines:

http_access allow all
http_access allow mynet
http_access allow localhost
http_access deny all

result [1.1]:
--------------------------
http://www1.mywebsite.ca/ works
http://www.mywebsite.ca/ http://www1.mywebsiteusa.com/ time out

ERROR
The requested URL could not be retrieved

While trying to retrieve the URL: http://www.mywebsite.ca/

The following error was encountered:

* Unable to forward this request at this time.

This request could not be forwarded to the origin server or to any
parent caches. The most likely cause for this error is that:

* The cache administrator does not allow this cache to make direct
connections to origin servers, and
* All configured parent caches are currently unreachable.

[1.3]
---------------------------------------------------------------------
ips instead of names

cache_peer_domain x.y.z.247 www.mywebsite.ca
cache_peer_domain x.y.z.248 www1.mywebsite.ca
cache_peer_domain x.y.z.249 www.mywebsiteusa.com
cache_peer_domain webmail webmail.mywebsite.ca

[result 1.3]
----------------------------------------------------------------------------
2009/05/20 11:07:35| squid.conf, line 1480: No cache_peer 'x.y.z.247'
2009/05/20 11:07:35| squid.conf, line 1481: No cache_peer 'x.y.z.248'
2009/05/20 11:07:35| squid.conf, line 1482: No cache_peer 'x.y.z.249'

http://www1.mywebsite.ca/ works
http://www.mywebsite.ca/ works
http://www1.mywebsiteusa.com/ goes to www.mywebsite.ca

[1.4]
-------------------------------------------------------------------------
acl acl_www_mywebsite dstdomain www.mywebsite.ca
acl acl_www1_mywebsite dstdomain www1.mywebsite.ca
acl acl_www_mywebsiteusa dstdomain www.mywebsiteusa.com
acl acl_webmail dstdomain webmail.mywebsite.ca

http_access allow acl_www_mywebsite
http_access allow acl_www1_mywebsite
http_access allow acl_www_mywebsiteusa
http_access allow acl_webmail

cache_peer_access www_mywebsite allow acl_www_mywebsite
cache_peer_access www1_mywebsite allow acl_www1_mywebsite
cache_peer_access www_mywebsiteusa allow acl_www_mywebsiteusa
cache_peer_access webmail allow acl_webmail

[result 1.4]
-----------------------------------------------------------------------
www1.mywebsite.ca works
http://www.mywebsite.ca/ http://www1.mywebsiteusa.com/ time out:

ERROR
The requested URL could not be retrieved

While trying to retrieve the URL: http://www.mywebsite.ca/

The following error was encountered:

* Connection to www_mywebsite Failed

The system returned:

(110) Connection timed out

The remote host or network may be down. Please try the request again.

Your cache administrator is root.

[1.5]
-------------------------------------------------------------------------
acl acl_www_mywebsite dstdomain www.mywebsite.ca
acl acl_www1_mywebsite dstdomain www1.mywebsite.ca
acl acl_www_mywebsiteusa dstdomain www.mywebsiteusa.com
acl acl_webmail dstdomain webmail.mywebsite.ca

http_access allow acl_www_mywebsite
http_access allow acl_www1_mywebsite
http_access allow acl_www_mywebsiteusa
http_access allow acl_webmail

cache_peer_access www_mywebsite allow acl_www_mywebsite
cache_peer_access www1_mywebsite allow acl_www1_mywebsite
cache_peer_access www_mywebsiteusa allow acl_www_mywebsiteusa
cache_peer_access webmail allow acl_webmail

#hosts_file none

[result 1.5]
-----------------------------------------------------------------------
www1.mywebsite.ca works
http://www.mywebsite.ca/ http://www1.mywebsiteusa.com/ time out:

ERROR
The requested URL could not be retrieved

While trying to retrieve the URL: http://www.mywebsite.ca/

The following error was encountered:

* Connection to www_mywebsite Failed

The system returned:

(110) Connection timed out

The remote host or network may be down. Please try the request again.

[2] Chris Robertson
-------------------------------------------------------------------------

cache_peer x.y.z.247 parent 80 0 no-query no-digest originserver
forceddomain=www.mywebsite.ca name=www_mywebsite
cache_peer x.y.z.248 parent 80 0 no-query no-digest originserver
forceddomain=www1.mywebsite.ca name=www1_mywebsite
cache_peer x.y.z.249 parent 80 0 no-query no-digest originserver
forceddomain=www.mywebsiteusa.com name=www_mywebsiteusa
cache_peer x.y.z.250 parent 80 0 no-query no-digest
forceddomain=webmail.mywebsite.ca originserver name=webmail

cache_peer_domain www_mywebsite www.mywebsite.ca
cache_peer_domain www1_mywebsite www1.mywebsite.ca
cache_peer_domain www_mywebsiteusa www.mywebsiteusa.com
cache_peer_domain webmail webmail.mywebsite.ca
hosts_file none

[result 2]
-------------------------------------------------------------------------
www1.mywebsite.ca works
http://www.mywebsite.ca/ http://www1.mywebsiteusa.com/ time out:

ERROR
The requested URL could not be retrieved

While trying to retrieve the URL: http://www.mywebsite.ca/

The following error was encountered:

* Unable to forward this request at this time.

This request could not be forwarded to the origin server or to any
parent caches. The most likely cause for this error is that:

* The cache administrator does not allow this cache to make direct
connections to origin servers, and
* All configured parent caches are currently unreachable.

Your cache administrator is root.

[2.1]
----------------------------------------------------------------------------
cache_peer x.y.z.247 parent 80 0 no-query no-digest originserver
forceddomain=www.mywebsite.ca name=www_mywebsite
cache_peer x.y.z.248 parent 80 0 no-query no-digest originserver
forceddomain=www1.mywebsite.ca name=www1_mywebsite
cache_peer x.y.z.249 parent 80 0 no-query no-digest originserver
forceddomain=www.mywebsiteusa.com name=www_mywebsiteusa
cache_peer x.y.z.250 parent 80 0 no-query no-digest
forceddomain=webmail.mywebsite.ca originserver name=webmail

acl acl_www_mywebsite_ip myip x.y.z.47
acl acl_www1_mywebsite_ip myip x.y.z.48
acl acl_www_mywebsiteusa_ip myip x.y.z.49
acl acl_webmail_ip myip x.y.z.250

cache_peer_access www_mywebsite allow acl_www_mywebsite_ip
cache_peer_access www_mywebsite deny all

cache_peer_access www1_mywebsite allow acl_www1_mywebsite_ip
cache_peer_access www1_mywebsite deny all

cache_peer_access www_mywebsiteusa allow acl_www_mywebsiteusa_ip
cache_peer_access www_mywebsiteusa deny all

cache_peer_access webmail allow acl_webmail_ip
cache_peer_access webmail deny all

hosts_file none

[result 2.1]
-------------------------------------------------------------------------
www1.mywebsite.ca works
http://www.mywebsite.ca/ http://www1.mywebsiteusa.com/ time out:

ERROR
The requested URL could not be retrieved

While trying to retrieve the URL: http://www.mywebsite.ca/

The following error was encountered:

* Unable to forward this request at this time.

This request could not be forwarded to the origin server or to any
parent caches. The most likely cause for this error is that:

* The cache administrator does not allow this cache to make direct
connections to origin servers, and
* All configured parent caches are currently unreachable.

Thanks.

Joaquin
Received on Wed May 20 2009 - 22:12:33 MDT

This archive was generated by hypermail 2.2.0 : Fri May 22 2009 - 12:00:01 MDT