[squid-users] Intermittent Caching During Benchmark Testing

From: Kent, Mr. John <kent.saic@dont-contact.us>
Date: Mon, 16 Jun 2003 14:54:07 -0700

Greetings,

Running Squid-2.5.STABLE3 on Linux as an accelerator with 32 redirector
processes.

Tested it using Apache Bench calling the Squid server
with its DNS name, it worked perfectly and viewing the access.log
saw that every hit was from the cache TCP_HIT:NONE overall Requests Per
Second
was 25.

I am planning on running multiple Squid Accelerators behind DNS round-robin.
So need to access the Squid server using its IP number. So I ran the
following test
./ab -n 800 -c 200 http://199.9.2.135:8080/tc_pages/tc_home.html

This time the access.log shows only intermittent hits:

199.9.2.65 - - [16/Jun/2003:13:09:22 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
32336 TCP_MISS:DIRECT
199.9.2.65 - - [16/Jun/2003:13:09:22 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
32336 TCP_MISS:DIRECT
199.9.2.65 - - [16/Jun/2003:13:09:22 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
32464 TCP_HIT:NONE
199.9.2.65 - - [16/Jun/2003:13:09:22 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
32336 TCP_MISS:DIRECT
199.9.2.65 - - [16/Jun/2003:13:09:22 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
32464 TCP_HIT:NONE
199.9.2.65 - - [16/Jun/2003:13:09:23 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
32336 TCP_MISS:DIRECT
199.9.2.65 - - [16/Jun/2003:13:09:23 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
32336 TCP_MISS:DIRECT
199.9.2.65 - - [16/Jun/2003:13:09:23 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
32336 TCP_MISS:DIRECT
199.9.2.65 - - [16/Jun/2003:13:09:23 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
32464 TCP_HIT:NONE
199.9.2.65 - - [16/Jun/2003:13:09:23 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
11103 TCP_MISS:DIRECT
199.9.2.65 - - [16/Jun/2003:13:09:23 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
12195 TCP_HIT:NONE
199.9.2.65 - - [16/Jun/2003:13:09:23 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
12313 TCP_HIT:NONE
199.9.2.65 - - [16/Jun/2003:13:09:23 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
20671 TCP_MISS:DIRECT
199.9.2.65 - - [16/Jun/2003:13:09:23 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
26151 TCP_MISS:DIRECT
199.9.2.65 - - [16/Jun/2003:13:09:23 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
15199 TCP_MISS:DIRECT
199.9.2.65 - - [16/Jun/2003:13:09:23 -0700] "GET
http://calamari.nrlmry.navy.mil:7777/tc_pages/tc_home.html HTTP/1.0" 200
4149 TCP_MISS:DIRECT
199.9.2.65 - - [16/Jun/2003:13:09:23

Overall Requests per second RPS was half the previous result now only 12.

So my question is, is there some reason why I get intermittent cache hits
from the same page when accessing Squid with the IP number vs. DNS name?
Is there something I can change in my config file to improve the situation?
My config file is below
Is this some artifact of the benchmarking test?

Thank you very much,

Mr. John Kent
Naval Research Laboratory
Monterey, CA

  # CONFIG FILE FOR WWW_SQUID

# THIS MUST BE AN IP ADDRESS! www.nrlmry.navy.mil will fail!!
http_port 199.9.2.135:8080
icp_port 0
httpd_accel_host virtual
httpd_accel_port 7777
#httpd_accel_port 80

# NOTE: the RUDE_ROBOTS_IP line is automatically written
# by the rude_robots.pl script which writes the line
# then restarts Squid by running squid -k reconfigure
# acl aclname src ip-address/netmask ... (clients IP address)
acl RUDE_IP src "/data/www/web/htdocs_dyn/squid/etc/Rude_Robots_IP.txt"
http_access deny RUDE_IP

hierarchy_stoplist /tc\_pages /cgi\-bin /sat\-bin /tc\-bin /focus\-bin /~
/goes\_cc /coamps\-reg

# A list of words which, if found in a URL, cause the object to
# be handled directly by this cache. In other words, use this
# to not query neighbor caches for certain objects. You may
# list this option multiple times.

# Since pages created dynamically by tc-bin and sat-bin have
# an expire time on them I DO want them cached - jk
#hierarchy_stoplist /cgi-bin /~ /goes\_cc /coamps\-reg

acl QUERY urlpath_regex THUMB\.jpg LATEST\.jpg Latest\.jpg swish dev \~
dev\-bin tc\-dev Mod\-dev training SAIC shared\-bin
cgi\-bin sat\-dev goes\_cc cc composer coamps\-reg wusage sys\-bin banner
aerosol Case\_
no_cache deny QUERY

cache_mem 32 MB

cache_dir ufs /users/webuser/www_squid/cache 100 16 256
cache_access_log /users/webuser/www_squid/logs/access.log
cache_log /users/webuser/www_squid/logs/cache.log

emulate_httpd_log on

pid_filename /users/webuser/www_squid/logs/squid.pid

#debug_options ALL,1,28,9

redirect_program /data/www/web/htdocs_dyn/squid/www_redirect.pl
#redirect_program /users/webuser/www_squid/dyn_conf/calamari_redirect.pl
redirect_children 32

# Cannot use this option to accelerate multiple back-end servers!
# TAG: redirect_rewrites_host_header
# By default Squid rewrites any Host: header in redirected
# requests. If you are running a accelerator then this may
# not be a wanted effect of a redirector.
#
#Default:
# redirect_rewrites_host_header on
redirect_rewrites_host_header on

acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255

acl acceleratedHost dst 199.9.2.134/255.255.255.255
199.9.2.108/255.255.255.255 199.9.2.48/255.255.255.255 199.9.2.69/255.255
.255.255 199.9.2.33/255.255.255.255 199.9.2.43/255.255.255.255
199.9.2.92/255.255.255.255 199.9.2.100/255.255.255.255 199.9.
2.101/255.255.255.255 199.9.2.102/255.255.255.255
199.9.2.103/255.255.255.255 199.9.2.44/255.255.255.255 199.9.2.109/255.255.
255.255 199.9.2.72/255.255.255.255

acl acceleratedPort port 7777 8888
acl myserver src 127.0.0.1/255.255.255.255
acl SSL_ports port 443 563
acl Safe_ports port 80 81 3128 7777 8888 8080 81 443 563
acl CONNECT method CONNECT
acl Methods method GET POST

http_access allow manager localhost
http_access allow manager myserver
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow all
http_access allow acceleratedHost acceleratedPort
http_access allow Methods
# And finally deny all other access to this proxy
http_access deny all

cache_mgr webmaster@nrlmry.navy.mil

# DO NOT RUN AS ROOT!
cache_effective_user webuser
cache_effective_group webgroup

# TAG: request_entities
# Squid defaults to deny GET and HEAD requests with request entities,
# as the meaning of such requests are undefined in the HTTP standard
# even if not explicitly forbidden.
#
# Set this directive to on if you have clients which insists
# on sending request entities in GET or HEAD requests.
#
request_entities on

# TAG: httpd_accel_with_proxy on|off
# If you want to use Squid as both a local httpd accelerator
# and as a proxy, change this to 'on'. Note however that your
# proxy users may have trouble to reach the accelerated domains
# unless their browsers are configured not to use this proxy for
# those domains (for example via the no_proxy browser configuration
# setting)
#
httpd_accel_with_proxy off

# TAG: httpd_accel_uses_host_header on|off
# HTTP/1.1 requests include a Host: header which is basically the
# hostname from the URL. Squid can be an accelerator for
# different HTTP servers by looking at this header. However,
# Squid does NOT check the value of the Host header, so it opens
# a big security hole. We recommend that this option remain
# disabled unless you are sure of what you are doing.
#
# However, you will need to enable this option if you run Squid
# as a transparent proxy. Otherwise, virtual servers which
# require the Host: header will not be properly cached.
# NOTE: Setting this to yes WILL break pages!

# JK 29-JAN-02: When this is on, within the Squid access.log
# squid GETS a domain name:
# ie. GET http://kauai.nrlmry.navy.mil:7777/.....
# When this is off squid GETS an IP
# ie GET http://199.9.2.69:7777/focus-docs/....
# NONE://199.9.2.108:8888
# TRICK: You can change an image in cache by changing this, hitting
# the image from a browser, then switching it back off

httpd_accel_uses_host_header on

# TAG: httpd_accel_single_host on|off

# server then set this to on. This causes Squid to forward the request
# to this server irregardles of what any redirectors or Host headers
# says.
#
# Leave this at off if you have multiple backend servers, and use a
# redirector (or host table or private DNS) to map the requests to the
# appropriate backend servers. Note that the mapping needs to be a
# 1-1 mapping between requested and backend (from redirector) domain
# names or caching will fail, as cacing is performed using the
# URL returned from the redirector.
#
# See also redirect_rewrites_host_header.
#
#Default:
# httpd_accel_single_host off

memory_pools on
cachemgr_passwd disable all

# IP for Helium
dns_nameservers 199.9.2.40

# Specifies the number of logfile rotations to make when you
# type 'squid -k rotate'. The default is 10, which will rotate
# with extensions 0 through 9. Setting logfile_rotate to 0 will
# disable the rotation, but the logfiles are still closed and
# re-opened. This will enable you to rename the logfiles
# yourself just before sending the rotate signal.
#
# Note, the 'squid -k rotate' command normally sends a USR1
# signal to the running squid process. In certain situations
# (e.g. on Linux with Async I/O), USR1 is used for other
# purposes, so -k rotate uses another signal. It is best to get
# in the habit of using 'squid -k rotate' instead of 'kill -USR1
# <pid>'.
#
#Default:
logfile_rotate 21

Mr. John Kent
SAIC Contractor
Naval Research Laboratory
Monterey, CA
Received on Mon Jun 16 2003 - 15:54:15 MDT

This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 17:17:23 MST