Re: [squid-users] Squid stops responding-LTSP and WinXP clients

Avinash Rao wrote:
> On Thu, Sep 10, 2009 at 10:54 AM, Avinash Rao <avinash.aol_at_gmail.com> wrote:
>> On Wed, Sep 9, 2009 at 5:45 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
>>> Avinash Rao wrote:
>>>> Dear all,
>>>>
>>>> I am sure this question would have been posted many times. I read a
>>>> few threads, but my requirement or setup is a bit different. I am
>>>> running Squid2.6stable18 on Ubuntu Server 8.04 Server 64-bit installed
>>>> on Sun Fire X4150 Server with 8GB RAM + 8 SAS HDD's - RAID 5 + 2 Quad
>>>> Core Intel Xeon Processors.
>>>>
>>>> I have both LTSP and WinXP clients using Squid. Many times i have
>>>> noticed, squid stops responding, the browser keeps trying to connect
>>>> and i don't see any error in cache.log or store.log. My only way out
>>>> is to restart squid.
>>> Usually means Squid is doing something with its on-disk storage.
>>>
>>>> I read through cache_mem, my current configuration is 100MB, i
>>>> changed it to 128MB, but squid couldn't start saying the cache_mem is
>>>> more than cache_disk size.
>>>>
>>>> Why does this happen and what is the recommended configuration for the
>>>> hardware i have?
>>> You are missing a cache_dir option. Which means Squid-2 will be using the
>>> default and very inefficient 100 MB ufs formatted storage.
>>>
>>> Squid-3.1 and later are the only releases of Squid where cache_dir can be
>>> fully removed from the config.
>>>
>>> Either define a cache_dir to use the disks or setup the "null" cache_dir
>>> type for memory-only storage.
>>>
>>>> squid.conf: Please donot compare this config to my other posts, as
>>>> this is on a different server and it has a very basic configuration.
>>>>
>>>> visible_hostname sunserver
>>>> hierarchy_stoplist cgi-bin ?
>>>> acl QUERY urlpath_regex cgi-bin \?
>>>> no_cache deny QUERY
>>> Drop 'no_cache' use instead:
>>> cache deny QUERY
>>>
>>>> hosts_file /etc/hosts
>>>> http_port 10.10.10.10:3128
>>>> refresh_pattern ^ftp: 1440 20% 10080
>>>> refresh_pattern ^gopher: 1440 0% 1440
>>>> refresh_pattern . 0 20% 4320
>>>>
>>>> acl all src 0.0.0.0/0.0.0.0
>>>> acl manager proto cache_object
>>>> acl localhost src 127.0.0.1/255.255.255.255
>>>> acl to_localhost dst 127.0.0.0/8
>>>> acl SSL_ports port 443 563
>>>> acl Safe_ports port 80 # http
>>>> acl Safe_ports port 21 # ftp
>>>> acl Safe_ports port 443 563 # https, snews
>>>> acl Safe_ports port 70 # gopher
>>>> acl Safe_ports port 210 # wais
>>>> acl Safe_ports port 1025-65535 # unregistered ports
>>>> acl Safe_ports port 280 # http-mgmt
>>>> acl Safe_ports port 488 # gss-http
>>>> acl Safe_ports port 591 # filemaker
>>>> acl Safe_ports port 631 # cups
>>>> acl Safe_ports port 777 # multiling http
>>>> acl Safe_ports port 901 # SWAT
>>>> acl Safe_ports port 993 # IMAP
>>>> acl Safe_ports port 587 # SMTP
>>>> acl Safe_ports port 22 # SSH
>>>> acl purge method PURGE
>>>> acl special_urls url_regex "/etc/squid/squid-noblock.acl"
>>>> acl extndeny url_regex -i "/etc/squid/blocks.files.acl"
>>>> acl malware_block_list url_regex -i "/etc/squid/malware_block_list.txt"
>>>> acl badurl url_regex -i teen orkut youtube sex mp3 mp4
>>>> acl lan src 10.10.10.0/24
>>>> acl stud ident_regex babu
>>>> acl download method GET
>>>> acl CONNECT method CONNECT
>>>> cache_mem 100 MB
>>>> ident_lookup_access allow all
>>>> deny_info http://malware.hiperlinks.com.br/denied.shtml malware_block_list
>>>> http_access allow manager localhost
>>>> http_access deny manager
>>>> http_access allow purge localhost
>>>> http_access allow special_urls
>>>> http_access deny extndeny download
>>>> http_access deny extndeny
>>>> http_access deny purge
>>>> http_access deny !Safe_ports
>>>> http_access deny CONNECT !SSL_ports
>>>> http_access deny badurl
>>>> http_access deny malware_block_list
>>>> http_access allow localhost
>>>> http_access allow lan
>>>> http_access deny all
>>>> http_reply_access allow all
>>>> icp_access allow all
>>>> coredump_dir /var/spool/squid
>>>>
>>>>
>>>> Many Thanks
>>>> Avinash
>>>
>>> --
>>> Please be using
>>> Current Stable Squid 2.7.STABLE6 or 3.0.STABLE18
>>> Current Beta Squid 3.1.0.13
>>>
>> Thank you all for the information. I read the documentation.
>> To begin with I have set cache_dir to 256MB and cache_mem also to 256
>> MB. Which takes the precedence? I guess the pages are stored/retrieved
>> from the RAM. Does this setting mean that, once 256MB is reached in
>> RAM it will start storing the pages in cache_dir?
>>
>> Thanks,
>> Avinash
>>
>
> Hi Guys,
>
> I just tested it, but it was pretty slow, the page was not loading
> only. Strange coz,
> the cache_mem is increased to 256MB and its still slow? How does this work?

Everything gets stuffed into cache_mem first.
  - objects which are labeled as too large for memory, or those
unlabeled which grow too large, get pushed to disk. This happens even if
not completely downloaded yet, which can be one cause of slowness on big
stuff.
  - objects which have not been used longest get pushed to disk when
more memory space is needed.
  - all memory objects are pushed to disk during shutdown, restart,
reconfigure.

> According to the squid Documentation, i recommended size for cache_dir
> is 7000 or 8000!
>
> I set cache_dir to null /tmp and i could see a lot of difference and
> it worked pretty well.

Seems to me that the issue was some things being sent to disk storage.
Combined with severe speed problems reading/waiting from that disk storage.

You said you have RAID5 so I'm not surprised its slow.
RAID and Squid do NOT play well together. Squid treats the disk as
random-access media using as many access threads as it can obtain in
parallel. Doing 4KB chunk operations.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE18
   Current Beta Squid 3.1.0.13