Re: [squid-users] Reverse Proxy URL rewriter/hostname scrubber

From: Seann Clark <>
Date: Mon, 19 Jan 2009 17:49:39 -0600

Amos Jeffries wrote:
>> All,
>> I am looking for a squid based solution for a problem I have. I
>> have a squid reverse proxy sitting in front of three web servers, set up
>> so I can test and QA each server by itself without a problem, and yet
>> still server to the www site as if they were one. While this works fine
>> inside my security boundary, it really breaks outside due to http being
>> blocked to the servers themselves.
>> The idea with the proxy is to go internet --> squid --> http farm
>> and the problem that comes about is that occasionally the servers will
>> response with hostname.tld instead of www.tld like they are supposed to.
>> Since I have it set up to be desirable both ways for my set up, this is
>> ok inside. I need to get my squid to grab those requests and re-write
>> them as they are served to the client. This is especially troublesome
>> for embedded generated code (JavaScript links, and silly stuff like
>> that). I have basic idea's on how to do this, but I haven't found either
>> a good step by step telling how to do this, or a good redirect client
>> that can handle something like this.
>> Any suggestions would help out a lot
>> Thanks,
>> Seann
> Squid really can't do this properly for replies.
> You need to find out why the servers are breaking their configuration. It
> may come down to replacing their software, but using broken apps is not a
> good idea.
> Assuming you are using the correct reverse-proxy configuration with no
> re-writes anywhere:
> Amos
My current squid set up for the reverse proxy (not sure if it is 100%
right, but when it was done, there was very little I could find on this,
aside from small entries in the wiki):

# $Id: squid.conf,v 1.1 2005/07/16 22:24:57 jmates Exp $
# This is the inital configuration file for the reverse proxy that will
be hosted on yukiko
# and will support HTTPS communication for the site external of the
network. It will be
# hardened with firewall software, and chrooting it as well.

http_port 80 accel
https_port 443 accel
cert=/etc/pki/tls/certs/haruhi.crt key=/etc/pki/tls/private/haruhi.key
icp_port 0
#htcp_port 0
# Security measure : Chroot the proccess (may require values?)
# disable usual block on cgi-bin and ? in URL, to avoid web robots
# adding ? to requests to bypass the proxy. Little cgi-bin and ? use on
# my site, so not a problem for me...
#hierarchy_stoplist cgi-bin ?
#acl QUERY urlpath_regex cgi-bin \?
#no_cache deny QUERY

cache_mem 256 MB
# cache_swap_low 90
# cache_swap_high 95

maximum_object_size 1024 KB
# minimum_object_size 0 KB

maximum_object_size_in_memory 64 KB

cache_replacement_policy lru
#cache_replacemnet_policy heap lfuda
memory_replacement_policy lru

cache_dir ufs /var/spool/squid/cache 5000 16 256

logfile_rotate 3

cache_log none
cache_store_log none

cache_access_log /var/log/squid/access_log
#cache_log /var/log/squid/cache_log
#cache_store_log /var/log/squid/store_log

emulate_httpd_log on
log_ip_on_direct on
log_mime_hdrs on

log_fqdn on

ftp_sanitycheck on

# use local caching name server, avoid /etc/hosts
hosts_file none
#redirect_program /usr/local/squirm/bin/squirm
#redirect_program /etc/squid/
#redirect_children 1
#redirect_program /etc/squid/rewriter/
#redirect_children 2
redirect_rewrites_host_header on
#Error code redirects
error_map 301
error_map 400
error_map 401
error_map 403
error_map 404
error_map 414
error_map 500
error_map 503
error_map 505

#location_rewritre_program /etc/squid/
# TODO needed??
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours

request_header_max_size 16 KB
# request_body_max_size 0 KB

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320

negative_ttl 1 minutes
negative_dns_ttl 1 minutes

# connect_timeout 1 minutes
# peer_connect_timeout 30 seconds
# read_timeout 10 minutes
# request_timeout 5 minutes
# persistent_request_timeout 2 minute
# half_closed_clients on

ident_timeout 1 seconds

shutdown_lifetime 17 seconds

acl manager proto cache_object
acl tsuki src
acl mariko src
acl hotaru src
acl haruhicm src
acl minazuki src
acl yuki-priv src
acl localhost src
acl to_localhost dst
acl all src
acl tsuki_sites dstdomain
#acl some_site dstdomain
acl bigsis urlpath_regex ^/bigsis
acl mh urlpath_regex ^/misterhouse
acl zm urlpath_regex ^/zm
acl base urlpath_regex ^/base
acl nulog urlpath_regex ^/nulog
acl squid_yuki urlpath_regex ^/squid
acl squid_yukiko urlpath_regex ^/yukiko-squid
acl music1 urlpath_regex ^/pitchfork
acl music2 urlpath_regex ^/phpMp
acl music3 urlpath_regex ^/phpMp2
acl jin urlpath_regex ^/jin
acl mmm urlpath_regex ^/mmm
acl lm urlpath_regex ^/Haruhi_sensors
acl lm2 urlpath_regex ^/Minazuki_sensors
acl amp urlpath_regex ^/amp
acl ups urlpath_regex ^/ups
acl fax urlpath_regex ^/fax
acl webmail urlpath_regex ^/webmail
acl otrs urlpath_regex ^/otrs
acl cacti urlpath_regex ^/cacti
acl tsukinokage urlpath_regex ^/tsukinokage_new
acl dnd urlpath_regex ^/dnd
acl cgi urlpath_regex ^/cgi-bin
acl dnd_upload urlpath_regex ^/dnd_upload
acl sysinfo urlpath_regex ^/phpsysinfo
acl SSL_ports port 443 563
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
#acl Safe_ports port 70 # gopher
#acl Safe_ports port 210 # wais
#acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 1025-65535 # unregistered ports
#acl Safe_ports port 488 # gss-http
#acl Safe_ports port 591 # filemaker
#acl Safe_ports port 777 # multiling http

http_access allow manager tsuki
http_access deny manager
http_access allow tsuki_sites
# Deny requests to unknown ports
http_access deny !Safe_ports
# Deny CONNECT to other than SSL ports
http_access deny CONNECT !SSL_ports

#server load balancing
#Directive hostname type proxy port icp
port options
#cache_peer parent/sib 80/443/3128
3130 stuff
cache_peer parent 80 0 no-query originserver name=haruhi
login=PASS round-robin Weight=5
cache_peer parent 80 0 no-query originserver name=minazuki
login=PASS round-robin Weight=1
#cache_peer parent 80 0 no-query originserver name=hikari
#cache_peer parent 80 0 no-query originserver name=akari
cache_peer_access haruhi allow cgi
cache_peer_access minazuki deny cgi
cache_peer_access haruhi allow sysinfo
cache_peer_access minazuki deny sysinfo
cache_peer_access haruhi allow dnd
cache_peer_access minazuki deny dnd
cache_peer_access haruhi allow dnd_upload
cache_peer_access minazuki deny dnd_upload
cache_peer_access haruhi allow otrs
cache_peer_access minazuki deny otrs
cache_peer_access haruhi allow tsukinokage
cache_peer_access minazuki deny tsukinokage
cache_peer_access haruhi allow cacti
cache_peer_access minazuki deny cacti
cache_peer_access haruhi allow webmail
cache_peer_access minazuki deny webmail
cache_peer_access haruhi allow amp
cache_peer_access minazuki deny amp
cache_peer_access haruhi allow ups
cache_peer_access minazuki deny ups
cache_peer_access minazuki allow bigsis
cache_peer_access haruhi deny bigsis
cache_peer_access minazuki allow fax
cache_peer_access haruhi deny fax
cache_peer_access minazuki allow lm2
cache_peer_access haruhi allow mmm
cache_peer_access minazuki deny mmm
cache_peer_access haruhi allow jin
cache_peer_access minazuki deny jin
cache_peer_access haruhi deny lm2
cache_peer_access haruhi allow lm
cache_peer_access minazuki deny lm
cache_peer_access minazuki allow mh
cache_peer_access haruhi deny mh
cache_peer_access minazuki allow zm
cache_peer_access haruhi deny zm
cache_peer_access haruhi allow base
cache_peer_access minazuki deny base
cache_peer_access haruhi allow nulog
cache_peer_access minazuki deny nulog
cache_peer_access haruhi allow squid_yuki
cache_peer_access minazuki deny squid_yuki
cache_peer_access haruhi allow squid_yukiko
cache_peer_access minazuki deny squid_yukiko
cache_peer_access haruhi allow music1
cache_peer_access minazuki deny music1
cache_peer_access haruhi allow music2
cache_peer_access minazuki deny music2
cache_peer_access haruhi allow music3
cache_peer_access minazuki deny music3
#http_access deny to_localhost

acl okdomains dstdomain
http_access deny !to_localhost
http_access allow all

# And finally deny all other access to this proxy
#http_access deny all

http_reply_access allow all

icp_access deny all
miss_access allow all

ident_lookup_access deny all


cache_effective_user squid

forwarded_for on

log_icp_queries off

snmp_port 3494
acl snmpharuhi snmp_community wintersnow
snmp_access allow snmpharuhi haruhicm
snmp_access deny all

# offline_mode off
coredump_dir /var/squid/core

pipeline_prefetch on

Received on Mon Jan 19 2009 - 23:50:11 MST

This archive was generated by hypermail 2.2.0 : Tue Jan 20 2009 - 12:00:07 MST