[squid-users] Re: slow browsing in centos 6.3 with squid 3 !! from Ahmad on 2013-02-25 (squid-users)

From: Ahmad <ahmed.zaeem_at_netstream.ps>
Date: Mon, 25 Feb 2013 00:18:47 -0800 (PST)

Amos Jeffries-2 wrote
> On 25/02/2013 12:30 a.m., Ahmad wrote:
>> hello ,
>> thanks Amos , ive modified the config file as u suggested .
>> after removing the raid 0 , ive noted a better performance .
>> =============================================================
>> in general , browsing speed is lower than the speed in the absence of
>> squid
>> , but any way it is acceptable and i wish to enhance it as i can !
>> ======================================================
>> As i mentioned in the beginning , i have an excellent hardware with about
>> 32
>> G ram.
>> but i have major problem in squid-guard !!
>> after sometime it begins to bypass!!!!!!
>> i searched to use dansguardian instead of squid-guard but it seems that
>> dansguardian is not compatible with tproxy !!===> seems as shook to me !
>> ==================================================
>>
>> i have pumped only 1000 users with about 150-180 M only !!!!
>> here is the log of squidguard !
>> ==============
>> 2013-02-24 06:25:32 [17282] Warning: Possible bypass attempt. Found
>> multiple
>> slashes where only one is expected:
>> http://surprises.tango.me/ts//assets/ayol_fairy_gingerbread_surprise_2-UI_VG_SELECTOR_PACK-android.zip
>
> Ah I see. SquidGuard is detecting what it reports as "bypass attempt".
>
> This is NOT squidguard being bypassed.
>
> There is a type of Web server attack *called* a "bypass attack" which
> was designed to use multiple slashes like // or ./ or ../ to trick
> simple URL matching security rules (like Squidguard appears to be using)
> into ignoring parts of the URL. Any pattern match regex which you are
> applying on the URL looking for the "http://" by ignoring the "http:"
> portion and identifying the "//" portion as the start will ignore the
> real domain name, attack login details, and maybe some of the path.
>
> However "//" is not necessarily a wrong patten. The author of the
> website determines what the URL syntax is, so if the web server the URL
> is supposed to be handled by can cope with it correctly that is a valid
> URL.
>
>> 2013-02-24 06:27:04 [17282] Warning: Possible bypass attempt. Found a
>> trailing dot in the domain name:
>> http://www.google.ps/xjs/_/js/s/sy15,gf,adnsp,wta,sy5,sy45,sy47,sy6,sy50,sy46,sy51,sy7,sy48,sy53,sy54,sy49,sy52,adct,ssi/rt=j/ver=OMt9IcC1O10.en_US./am=CA/d=0/sv=1/rs=AItRSTOekKHDXRJiLDzqcQkCe4C3pVWkbw
>
> "Trailing dot" ??
>
> Oh I see. .http://.... C1O10.en_US./
>
> Whatever URL match squidGuard is testing there is *VERY* broken. Only
> [a-zA-Z0-9\-\.\:] are permitted characters in domain names (or raw-IP
> whch can also be there). squidGuard pattern is currently is allowing _ ,
> / = and probably # and ? as well I guess.
> You need to fix that pattern *immediately* regardless of whatever else
> you do about squidGuard.
>
>> [root_at_squid ~]#
>> ==============================
>> here is a sample of cache.log file:
>> {Accept: */*
>> Content-Type: application/x-www-form-urlencoded
>> 2013/02/24 06:24:18| WARNING: HTTP header contains NULL characters
>> {Accept:
>> */*
>> Content-Type: application/x-www-form-urlencoded}
>> NULL
>> {Accept: */*
>> Content-Type: application/x-www-form-urlencoded
>> 2013/02/24 06:24:18| WARNING: HTTP header contains NULL characters
>> {Accept:
>> */*
>> Content-Type: application/x-www-form-urlencoded}
>> NULL
>> {Accept: */*
>> Content-Type: application/x-www-form-urlencoded
>> 2013/02/24 06:24:18| WARNING: HTTP header contains NULL characters
>> {Accept:
>> */*
>> Content-Type: application/x-www-form-urlencoded}
>> NULL
>> {Accept: */*
>> Content-Type: application/x-www-form-urlencoded
>> 2013/02/24 06:24:18| WARNING: HTTP header contains NULL characters
>> {Accept:
>> */*
>> Content-Type: application/x-www-form-urlencoded}
>> NULL
>> {Accept: */*
>> Content-Type: application/x-www-form-urlencoded
>> 2013/02/24 06:24:41| clientProcessRequest: Invalid Request
>> 2013/02/24 06:25:00| clientProcessRequest: Invalid Request
>> 2013/02/24 06:25:04| clientProcessRequest: Invalid Request
>> 2013/02/24 06:25:07| clientProcessRequest: Invalid Request
>> 2013/02/24 06:25:09| helperHandleRead: unexpected reply on channel 0 from
>> redirector #1 ''
>
> The squidGuard helper is sending Squid more lines of response than Squid
> sent lines of requests.
> It looks like something is causing an extra newline at the end of a
> response.
>
> The above happening will cause that squidGuard helper to be killed and a
> new one to be started. This process will slow down your Squid with a
> small pause as the new helper is started. If it happens often that could
> be a large part of your speed problem.
>
>
> Amos

Hi Mr Amos ,
thanks very much for explanation .
thanks Marcus ,

so ,

you mentioned that i have to fix the ... and // in squidguard !! how could
i fix it ??!!!!!!

i want to say something !

ive removed squid 1.4 and installed squidguard 1.5 beta version .

after that ,
no bypass happened :)
i mean that it was seem to be problem of squidguard .
i read that there is a bugs in squidguard in bypass , and i found squid 1.5
is better .
i pumped 2000 users to squid with BW 200M and no by pass occured

this is one issue ,

now lets return to the issue of slow browsing ,
agian , the browsing is not very bad , but it is acceptable anyway and less
quality than in absense of squid .

i dont know if it was because of my hardsiks !!
my disks are as bellow :
hd1==>ssd with 180 G as operating system
hd2==>sata with 560 G as /cache1 storage
hd3==>sata with 560 G as /cache2 storage
hd4===>sata with 560 G as /cache3 storage

now i dont know if i need more hardiks additional to hd2, hd3 , hd4 ?
or i need to replace them by ssd ?
or i need to use another file system to enhacne the speed ?

You may advice me Mr Amos about the best choice :)

cache.log

*NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:19| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:20| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/25 03:12:21| WARNING: HTTP header contains NULL characters {Accept:
*/*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded*

======================================

do i need to increase the redirector in squidguard ???
here is squid.conf file :

*[root_at_squid ~]# cat /etc/squid/squid.conf
#
#
# squid Config By "Drvirus"
#
###################
acl all src all
acl manager proto cache_object
acl localnet src 192.168.1.0/24 x.x.x.x/16 x.x.x.x/16
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 590 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
################################
visible_hostname squid
coredump_dir /var/spool/squid
####squidguard###################
redirect_program /usr/local/squidguard5/bin/squidGuard -c
/etc/squidguard.conf
redirector_bypass on
url_rewrite_children 200
###############################
cache_effective_user squid
cache_effective_group squid
##############################
#Recommended minimum configuration:
# Only allow cachemgr access from localhost
http_access allow manager localhost
http_access allow localnet
http_access deny manager
# Deny requests to unknown ports
http_access deny !Safe_ports
# Deny CONNECT to other than SSL ports
http_access deny CONNECT !SSL_ports
# And finally deny all other access to this proxy
http_access deny all
#Allow ICP queries from everyone
icp_access allow all
#######################################
access_log /var/log/squid/access.log
cache_dir aufs /cache1 500000 32 256
cache_dir aufs /cache2 500000 32 256
cache_dir aufs /cache3 500000 32 256
cache_mem 20000 MB
##########################
http_port 127.0.0.1:3128
http_port x.x.x.x:x.x
http_port 3128
http_port 3129 tproxy
########### Performance Related Config:
relaxed_header_parser on
vary_ignore_expire on
##########################################
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
###########################################
ipcache_size 2048
ipcache_low 98
ipcache_high 99
memory_pools off
pipeline_prefetch on
############################################
httpd_suppress_version_string on
server_persistent_connections on
client_persistent_connections on
pconn_timeout 2 minutes
persistent_request_timeout 1 minute
###########################################
########### WCCP2 Config#############
wccp2_router x.x.x.x
wccp_version 2
wccp2_forwarding_method 2
wccp2_return_method 2
#wccp2_assignment_method mask
wccp2_service dynamic 80
wccp2_service_info 80 protocol=tcp flags=src_ip_hash priority=240 ports=80
wccp2_service dynamic 90
wccp2_service_info 90 protocol=tcp flags=dst_ip_hash,ports_source
priority=240 ports=80
##########################################
###########################################
#default option
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
#################################################
forwarded_for on
max_filedescriptors 65536
max_open_disk_fds 65536
relaxed_header_parser on
reload_into_ims on
client_lifetime 15 minutes
read_timeout 5 minutes
request_timeout 1 minutes
ie_refresh on
ignore_expect_100 on
vary_ignore_expire on
###############################
################################
httpd_suppress_version_string on
server_persistent_connections on
client_persistent_connections on
pconn_timeout 2 minutes
persistent_request_timeout 1 minute
shutdown_lifetime 20 seconds
#############################
cache_swap_low 98
cache_swap_high 99
cache_replacement_policy heap LFUDA
minimum_object_size 0
maximum_object_size 130 MB
###############################*

with my best regards

--
View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/slow-browsing-in-centos-6-3-with-squid-3-tp4658635p4658697.html
Sent from the Squid - Users mailing list archive at Nabble.com.

Received on Mon Feb 25 2013 - 08:18:51 MST

This archive was generated by hypermail 2.2.0 : Mon Feb 25 2013 - 12:00:05 MST