Re: [squid-users] SQUID makes few slow connections (see video)

On Tue, 1 Jun 2010 23:50:35 +0200, Tytus Rogalewski <tytanick_at_gmail.com>
wrote:
> I thik that this is something to do it with many requests, because
> half of images are loaded imiedietly (with hits MISS) , and rest are
> loaded after few seconds (also MISSED) - missed because i have erased
> cache. and is page is cached - it works fine. Only NEW requests are
> working slowly

Sounds like normal network delays. Horribly long though one there being
12s.

As asked earlier are you forward-proxying or reverse-proxying with this
Squid?
ie, are you providing access to a local web server where these pages are
hosted?
 or just trying to get internal clients good access to the external site?

<snip earlier provided traces>

>> ===========================
>> SQUID SETTINGS
>>
>> http_port 8123 transparent
>> cache_mgr tytan..._at_gmail.com
>> visible_hostname sky-link.net
>> error_directory /usr/share/squid/errors/Polish_clean
>>
>> cache_mem 1500 MB # ilosc ramu
>> memory_pools on
>> memory_pools_limit 100 MB
>> cache_swap_low 94
>> cache_swap_high 98
>> fqdncache_size 51200
>> ipcache_size 51200
>> ipcache_low 90
>> ipcache_high 95
>> pipeline_prefetch on

mumble. This _might_ be affecting lag time since pipelining accepts a
second request and starts it's timers before the first request has finished
and the second can actually start being handled.
Squid will still be able to handle a small number of client-pipelined
requests with this turned off.

>> minimum_object_size 0 KB
>> maximum_object_size 50000 KB
>> maximum_object_size_in_memory 512 KB
>> half_closed_clients on
>> shutdown_lifetime 2 seconds
>> #ident_timeout 1 seconds
>> cache_replacement_policy heap LFUDA
>> memory_replacement_policy heap lru
>> #cache_dir aufs /mnt/dysk/squid 100000 256 256
>> cache_dir aufs /mnt/dysk/squid 20000 32 256
>> #quick_abort_min 200 KB
>> #quick_abort_max 200 KB
>> quick_abort_min 0 KB
>> quick_abort_max 0 KB
>>
>> ### OPCJE ###
>> icp_port 0
>> htcp_port 0
>> icp_access deny all
>> htcp_access deny all
>> snmp_port 0
>> snmp_access deny all
>>
>> ### LOGI ###
>> access_log /var/log/squid/access.log
>> cache_log /var/log/squid/cache.log
>> cache_store_log /var/log/squid/store.log

store.log is not very useful for most and wastes disk IO resources.
You can probably set this to:
  cache_store_log none

>> log_fqdn on # pelne nazwy domen w logach

Forces rDNS lookups to log things. This can add lag.

>> #debug_options ALL,1
>>
>> ie_refresh on
>> cache_effective_user squid
>> cache_effective_group squid
>> negative_ttl 1 minutes
>> positive_dns_ttl 24 hours
>> negative_dns_ttl 1 minutes

Storing DNS results for longer than the authoritive website allows can
lead you to the lag problem, if the web server IP has changed and Squid
struggles to find a bad route to the old IP.

>>
>> ## ACL
>> acl manager proto cache_object
>> acl localhost src 127.0.0.1/255.255.255.255
>> acl sky-link src 192.168.0.0/255.255.255.0 #Nasza LAN
>> acl SSL_ports port 443 563
>> acl Safe_ports port 21 80 442 563 70 210 280 488 591 777 1025-65535
>> acl CONNECT method CONNECT
>> #acl MCONN maxconn 300
>> # Youtube ACL
>> acl youtube_domains
>> dstdomain .youtube.com .googlevideo.com .ytimg.com .wrzuta.pl
>>
>> ### ADL Blokada Reklam ###
>> acl allowlist_urlregex url_regex "/etc/squid/acl/
>> allowlist_urlregex.acl"
>> acl allowlist_dstdomain dstdomain "/etc/squid/acl/
>> allowlist_dstdomain.acl"
>> acl denylist_urlregex url_regex "/etc/squid/acl/denylist_urlregex.acl"
>> acl denylist_dstdomain dstdomain "/etc/squid/acl/
>> denylist_dstdomain.acl"
>>
>> ## HTTP ACCESS
>> http_access allow allowlist_dstdomain
>> http_access allow allowlist_urlregex
>> http_access deny denylist_urlregex
>> http_access deny denylist_dstdomain

 Optimization Tip #1:
    place regex rules as late as possible.

 Meaning:
  http_access deny denylist_dstdomain
  http_access deny denylist_urlregex

>> http_access allow sky-link
>> http_access allow manager localhost
>> http_access deny manager
>> http_access deny CONNECT !SSL_ports
>> http_access deny !Safe_ports
>> http_access deny youtube_domains

You access lists can be re-ordered to gain a fair bit of speed on some
requests. Not sure if its related to the ones showing you problems
though...

  # These rules are the security boundaries preventing nasty abusive
behaviour
  # by your clients intentionally or when infected.
  # They also form the second level of DDoS protection for your Squid.
  http_access allow manager localhost
  http_access deny manager
  http_access deny CONNECT !SSL_ports
  http_access deny !Safe_ports

I have to assume that since you are using "transparent" option, your proxy
is forward-only. (It's a giant security breach of your network to do
reverse-proxy that way.)

  http_access deny !sky-link
  http_access allow allowlist_dstdomain
  http_access allow allowlist_urlregex
  http_access deny denylist_dstdomain
  http_access deny denylist_urlregex
  http_access allow sky-link
  http_access deny all

Note how the youtube rule is now missing, it was not having any affect on
the LAN as defined by sky-link ACL.

>>
>> #tcp_outgoing_address 79.190.121.50
>> #zph_tos_local 8
>> #zph_tos_peer 0
>> #zph_tos_parent off
>>
>> #vary_ignore_expire on
>> #relaxed_header_parser on
>> #server_persistent_connections off
>>
>> request_header_max_size 100 KB
>> refresh_pattern -i \.(gif|png|jpg|jpeg|ico|bmp)$ 10080 90% 43200
>> refresh-ims
>> refresh_pattern -i \.(iso|avi|wav|ogg|mp3|mp4|mpg|mpeg|swf|flv|x-flv)$
>> 10080 90% 43200 refresh-ims
>> refresh_pattern -i \.(deb|rpm|exe|zip|tar|gz|bz2|tgz|ram|rar|bin|ppt|
>> doc|tiff)$ 10080 90% 43200 refresh-ims
>> refresh_pattern windowsupdate.com/.*\.(cab|exe|dll) 10080
>> 90% 43200 refresh-ims
>> refresh_pattern download.microsoft.com/.*\.(cab|exe|dll) 10080
>> 90% 43200 refresh-ims
>> refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll) 10080
>> 90% 43200 refresh-ims
>> refresh_pattern symantecliveupdate.com/.*\.(zip|exe) 10080
>> 90% 43200 refresh-ims
>> refresh_pattern windowsupdate.com/.*\.(cab|exe) 10080
>> 90% 43200 refresh-ims
>> refresh_pattern download.microsoft.com/.*\.(cab|exe) 10080
>> 90% 43200 refresh-ims
>> refresh_pattern avast.com/.*\.(vpu|vpaa) 10080
>> 90% 43200 refresh-ims

You are missing the refresh_pattern that ensures your Squid behaves
correctly when handling broken dynamic pages:

  refresh_pattern -i (/cgi-bin/|\?) 0 0% 0

>> refresh_pattern . 0
>> 90% 43200 refresh-ims
>>

Amos
Received on Wed Jun 02 2010 - 00:11:29 MDT