Re: Fwd: [squid-users] Request Entity Too Large Error in Squid Reverse Proxy

From: Amos Jeffries <squid3_at_treenet.co.nz>
Date: Fri, 15 Aug 2014 04:16:45 +1200

On 15/08/2014 12:59 a.m., Robert Cicerelli wrote:
> On 8/14/2014 8:10 AM, Amos Jeffries wrote:
>> If you can provide your squid.conf it would be really helpful
>> understanding this. Amos
> I think the terminology is confusing because it's the terminology used
> in the pfsense box that squid is running on. Nevertheless, squid.conf is
> below:
>
> ====== squid.conf starts below ========
>
> http_port 10.10.14.1:3128
> icp_port 7
> dns_v4_first off

NP: not necessary. "off" is the default of dns_v4_first.

> pid_filename /var/run/squid.pid
> cache_effective_user proxy
> cache_effective_group proxy
> error_default_language en
> icon_directory /usr/pbi/squid-i386/etc/squid/icons
> visible_hostname localhost
> cache_mgr admin_at_localhost

Set visible_hostname correctly to an externally accessible hostname.

> access_log /var/squid/logs/access.log
> cache_log /var/squid/logs/cache.log
> cache_store_log none
> sslcrtd_children 0
> logfile_rotate 1
> shutdown_lifetime 3 seconds
> # Allow local network(s) on interface(s)
> acl localnet src 10.10.14.0/24
> uri_whitespace strip
>
> acl dynamic urlpath_regex cgi-bin \?
> cache deny dynamic

You may want to reconsider that. Squid since 2.6 are erfectly capable of
caching dynamic content correctly provided you add the refresh_pattern
rule for (/cgi-bin/|\?) in the right place.

> cache_mem 2000 MB
> maximum_object_size_in_memory 32 KB
> memory_replacement_policy heap GDSF
> cache_replacement_policy heap LFUDA
> cache_dir ufs /var/squid/cache 500 16 256
> minimum_object_size 0 KB
> maximum_object_size 4 KB
> offline_mode offcache_swap_low 90
> cache_swap_high 95
>
> # No redirector configured
>
>
> #Remote proxies
>
>
> # Setup some default acls
> acl allsrc src all
> acl localhost src 127.0.0.1/32
> acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 3128
> 1025-65535
> acl sslports port 443 563
> acl manager proto cache_object
> acl purge method PURGE
> acl connect method CONNECT
>
> # Define protocols used for redirects
> acl HTTP proto HTTP
> acl HTTPS proto HTTPS
>
> http_access allow manager localhost
>
> http_access deny manager
> http_access allow purge localhost
> http_access deny purge
> http_access deny !safeports
> http_access deny CONNECT !sslports
>
> # Always allow localhost connections
> http_access allow localhost
>
> quick_abort_min 0 KB
> quick_abort_max 0 KB

All of these...

> request_body_max_size 0 KB
> delay_pools 1
> delay_class 1 2
> delay_parameters 1 -1/-1 -1/-1
> delay_initial_bucket_level 100
> # Throttle extensions matched in the url
> acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
> delay_access 1 allow throttle_exts
> delay_access 1 deny allsrc

... do nothing. Except, the delay_pools still involve 32-bit limits so
may also be your issue if it is 32-bit related.

>
> # Reverse Proxy settings
> http_port 75.145.82.58:80 accel defaultsite=deeztek.com vhost
> https_port 75.145.82.58:443 accel
> cert=/usr/pbi/squid-i386/etc/squid/53dfccd7cbb37.crt
> key=/usr/pbi/squid-i386/etc/squid/53dfccd7cbb37.key
> defaultsite=deeztek.com vhost
> #
> cache_peer 10.10.14.254 parent 443 0 proxy-only no-query no-digest
> originserver login=PASS round-robin ssl sslflags=DONT_VERIFY_PEER
> front-end-https=auto name=rvp_webserver.deeztek.com
>
> #
> cache_peer 10.10.14.201 parent 443 0 proxy-only no-query no-digest
> originserver login=PASS round-robin ssl sslflags=DONT_VERIFY_PEER
> front-end-https=auto name=rvp_owa.deeztek.com
>
> #
> cache_peer 10.10.14.251 parent 458 0 proxy-only no-query no-digest
> originserver login=PASS round-robin ssl sslflags=DONT_VERIFY_PEER
> front-end-https=auto name=rvp_cloud.deeztek.com
>
> #
> cache_peer 10.10.14.238 parent 443 0 proxy-only no-query no-digest
> originserver login=PASS round-robin ssl sslflags=DONT_VERIFY_PEER
> front-end-https=auto name=rvp_ewa.deeztek.com
>
> #
> cache_peer 10.10.14.250 parent 443 0 proxy-only no-query no-digest
> originserver login=PASS round-robin ssl sslflags=DONT_VERIFY_PEER
> front-end-https=auto name=rvp_mail.deeztek.com
>
> #
> cache_peer 10.10.14.254 parent 80 0 proxy-only no-query no-digest
> originserver login=PASS round-robin name=rvp_admin.grubbcontractors.com
>

Note that "round-robin" peer selection does not exactly jive well with
explicit cache_peer_access restricting each peer to only accepting
certain domains traffic. The access rules make each round-robin group a
set of 1 peer.

Okay lets simplify these ACLs ...

> acl rvm_deeztek.com url_regex -i ^https://secure.deeztek.com/.*
> acl rvm_deeztek.com url_regex -i ^https://www.deeztek.com/.*
> acl rvm_deeztek.com url_regex -i ^https://forums.deeztek.com/.*
> acl rvm_deeztek.com url_regex -i ^https://deeztek.com/.*
> acl rvm_OWASSL url_regex -i ^https://owa.deeztek.com/.*
> acl rvm_OWASSL url_regex -i ^https://hdgexchange.deeztek.com/.*
> acl rvm_OWASSL url_regex -i ^https://activesync.deeztek.com/.*
> acl rvm_OWASSL url_regex -i ^https://autodiscover.deeztek.com/.*
> acl rvm_OWASSL url_regex -i ^https://autodiscover.mydirectmail.net/.*
> acl rvm_EWASSL url_regex -i ^https://ewa.deeztek.com/.*
> acl rvm_MAILSSL url_regex -i ^https://mail.deeztek.com/.*
> acl rvm_visionexperts.com url_regex -i ^https://www.visionexperts.com/.*
> acl rvm_visionexperts.com url_regex -i ^https://visionexperts.com/.*
> acl rvm_visionexperts.com url_regex -i ^https://secure.visionexperts.com/.*
> acl rvm_grubbcontractors.com url_regex -i
> ^https://www.grubbcontractors.com/.*
> acl rvm_grubbcontractors.com url_regex -i
> ^https://bids.grubbcontractors.com/.*
> acl rvm_grubbcontractors.com url_regex -i ^https://grubbcontractors.com/.*
> acl rvm_admin.grubbcontractors.com url_regex -i
> ^https://admin.grubbcontractors.com/.*

1) Make all of the above into dstdomain ACL type.

Example:
  acl rvm_grubbcontractors.com dstdomain grubbcontractors.com

2) add HTTP/HTTPS ACL test to the cache_peer_access below ...

 - for each peer which is required to accept HTTPS-only start its rule
list with:
     cache_peer_access ... deny !HTTPS

 - for each peer which is required to accept HTTP-only start its rule
list with:
     cache_peer_access ... deny !HTTP

> cache_peer_access rvp_webserver.deeztek.com allow rvm_deeztek.com
> cache_peer_access rvp_owa.deeztek.com allow rvm_OWASSL
> cache_peer_access rvp_ewa.deeztek.com allow rvm_EWASSL
> cache_peer_access rvp_mail.deeztek.com allow rvm_MAILSSL
> cache_peer_access rvp_webserver.deeztek.com allow rvm_visionexperts.com
> cache_peer_access rvp_webserver.deeztek.com allow rvm_grubbcontractors.com
> cache_peer_access rvp_admin.grubbcontractors.com allow
> rvm_admin.grubbcontractors.com
> cache_peer_access rvp_webserver.deeztek.com deny allsrc
> cache_peer_access rvp_owa.deeztek.com deny allsrc
> cache_peer_access rvp_ewa.deeztek.com deny allsrc
> cache_peer_access rvp_mail.deeztek.com deny allsrc
> cache_peer_access rvp_webserver.deeztek.com deny allsrc
> cache_peer_access rvp_webserver.deeztek.com deny allsrc
> cache_peer_access rvp_admin.grubbcontractors.com deny allsrc

Also, you may as well remove the "allsrc" from the above lines. No error
page is generated directly by these access controls. The default "cannot
connect" will be if none are selected.
 Just use "all" here instead.

It may help you manage what peer is doing what by grouping the rules for
each peer together in a sequential bunch.

> never_direct allow rvm_deeztek.com
> never_direct allow rvm_OWASSL
> never_direct allow rvm_EWASSL
> never_direct allow rvm_MAILSSL
> never_direct allow rvm_visionexperts.com
> never_direct allow rvm_grubbcontractors.com
> never_direct allow rvm_admin.grubbcontractors.com
> http_access allow rvm_deeztek.com
> http_access allow rvm_OWASSL
> http_access allow rvm_EWASSL
> http_access allow rvm_MAILSSL
> http_access allow rvm_visionexperts.com
> http_access allow rvm_grubbcontractors.com
> http_access allow rvm_admin.grubbcontractors.com
>
> deny_info TCP_RESET allsrc
>
> # Custom options
>
>
> # Setup allowed acls
> # Allow local network(s) on interface(s)
> http_access allow localnet
> # Default block all to be sure
> http_access deny allsrc
>
> ====== squid.conf ends above ========
>

HTH
Amos
Received on Thu Aug 14 2014 - 16:17:18 MDT

This archive was generated by hypermail 2.2.0 : Thu Aug 14 2014 - 12:00:05 MDT