Re: [squid-users] trouble caching some url

 On Tue, 15 Nov 2011 09:43:45 -0500, Sergey Yazikov wrote:
> Hi, I'm wondering if anyone can help me.
>
> I'm unable to cache the following URL and there is a HTTP header:
>
>
> http://10.10.202.126/services/json?method=media.getFiles&filters[moderationStatus]=notdenied&filters[parentid]=5050233&filters[context]=comment&noCache=0&sort=upload+DESC&limit=5&start=0&fields[0]=filetype&fields[1]=uid&fields[2]=user_name&fields[3]=votecount&fields[4]=author&fields[5]=upload&fields[6]=message&fields[7]=publicUrl&fields[8]=status&fields[9]=avatar&fields[10]=user_firstname&fields[11]=user_lastname&vhost=123&APIKEY=19b3b6ffe399eb681aa3372cc2cbd6a2
>
> GET
>
> /services/json?method=media.getFiles&filters[moderationStatus]=notdenied&filters[parentid]=5050233&filters[context]=comment&noCache=0&sort=upload+DESC&limit=5&start=0&fields[0]=filetype&fields[1]=uid&fields[2]=user_name&fields[3]=votecount&fields[4]=author&fields[5]=upload&fields[6]=message&fields[7]=publicUrl&fields[8]=status&fields[9]=avatar&fields[10]=user_firstname&fields[11]=user_lastname&vhost=123&APIKEY=19b3b6ffe399eb681aa3372cc2cbd6a2
> HTTP/1.1
> Host: 10.10.202.126
> User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:8.0) Gecko/20100101
> Firefox/8.0
> Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Accept-Language: en-us,en;q=0.8,fr-ca;q=0.5,ru;q=0.3
> Accept-Encoding: gzip, deflate
> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
> DNT: 1
> Connection: keep-alive
> Cookie: SABRE_ID=8961c05907e412b9bdcfcb3a2364b31a
>
> HTTP/1.0 200 OK
> Date: Tue, 15 Nov 2011 14:37:43 GMT
> Server: Apache
> Accept-Ranges: bytes
> P3P: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND
> CNT"
> X-FM: ip-10-85-159-13
> Expires: Thu, 19 Nov 1981 08:52:00 GMT

 The object being returned to squid expired in 1981. It should not be
 cached. Reverse-proxy MAY ignore this provided the max-stale or max-age
 values are sent by the server.

> Cache-Control: no-store, no-cache, must-revalidate, post-check=0,
> pre-check=0

 "no-store" - the object MUST NOT be stored anywhere other than the
 backend server.

 "must-revalidate" - every new request MUST contact the backend server
 for new data or approval before re-using this object.

 "no-cache" - see must-revalidate.

 "post-check=0, pre-check=0" - invalid.

> Pragma: no-cache
> Content-Type: application/json
> X-Cache: MISS from ip-10-85-159-13.ec2.internal, MISS from
> fmgateway-test.testdmz.pelmorex.com
> X-Cache-Lookup: MISS from ip-10-85-159-13.ec2.internal:80, MISS from
> fmgateway-test.testdmz.pelmorex.com:80
> Via: 1.1 ip-10-85-159-13.ec2.internal:80 (squid), 1.0
> fmgateway-test.testdmz.pelmorex.com (squid/3.1.4)
> Connection: close
>
> Here is the squid.conf:
>
> [root_at_fmgateway-test squid]# grep -v ^# /etc/squid/squid.conf
> cache_mgr netops_at_pelmorex.com
> hosts_file /etc/hosts
> visible_hostname fmgateway-test.testdmz.pelmorex.com
> cache_effective_user squid
> cache_effective_group squid
> cache_dir ufs /var/spool/squid 4096 16 256
> cache_mem 500 MB
> maximum_object_size_in_memory 128 KB
> minimum_expiry_time 120 seconds
> read_ahead_gap 30 KB
> negative_ttl 1 seconds
> forward_timeout 30 seconds
> connect_timeout 30 seconds
> read_timeout 30 seconds
> request_timeout 30 seconds

 NP: connect_timeout being the same as forward_timeout means that
 alternative connections cannot be made within forward_timeout.

> persistent_request_timeout 1 minute
> client_lifetime 20 hours
>
> icp_port 3130
> http_port 80 accel defaultsite=api.filemobile.com ignore-cc
> cache_peer api.filemobile.com parent 80 0 no-query originserver
> name=myAccel
> acl our_sites dstdomain api.filemobile.com
> http_access allow our_sites
> cache_peer_access myAccel allow all
>
> acl intsrc src 10.10.6.0/23 10.10.202.0/24 10.10.201.0/24
> acl intdst dst 10.10.6.0/23 10.10.202.0/24 10.10.201.0/24
> acl apache rep_header Server ^Apache
> http_reply_access allow all
> url_rewrite_host_header off
> acl CONNECT method CONNECT
> acl PURGE method PURGE
> acl POST method POST
> acl QUERY urlpath_regex storage
> acl JS urlpath_regex js
> no_cache deny QUERY
> no_cache deny POST
> no_cache deny JS

 NOTE: "no_cache" was renamed to "cache" long ago. Remove the "no_" part
 of all those lines.

 Also, "cache deny JS", matches the letters 'j' followed by 's' anywhere
 in the URL path. For example in '/services/json?'

 Also, POST method is never cacheable. You can optimize your cache ACLs
 a bit by removing "no_cache deny POST" entirely.

> http_access allow our_sites

 You have "http_access allow our_sites" already listed with
 cache_peer_access.

> http_access allow all
> http_access allow intsrc
> http_access allow intdst
> http_access allow PURGE intsrc
> http_access allow PURGE all

 Since you only have one http_port, which is configured to set the
 domain on all requests to "api.filemobile.com". The earlier ACL test
 "allow our_sites" will always match. Making these http_access lines
 completely useless. You can remove them, and the other http_access lines
 below.

> icp_access allow all
> acl manager proto cache_object
> acl localhost src 127.0.0.1/32
> acl internal src 10.10.201.60/32
> acl sergey src 10.10.6.164/32
> acl dennis src 10.10.6.176/32
> http_access allow manager localhost
> http_access allow manager internal
> cachemgr_passwd 6twN+9Mtl all

 NOTE: you had probably better change that password quickly. This is a
 public mailing list.

> request_header_access Via allow all
> request_header_access Age deny all
> request_header_access X-Cache deny all
> request_header_access Etag deny all

 ETag is a seriously important header to prevent objects with different
 binary representations being mixed up. This is a bad idea.

> request_header_access Expires deny all

 "Expires:" is not a request header.

>
>
> refresh_pattern -i ^http.*filemobile.com\/services\/php 40 90% 50
> override-expire override-lastmod ignore-reload reload-into-ims
> ignore-no-cache
>
> refresh_pattern -i ^http.*filemobile.com\/services\/json 10 90% 10
> override-expire override-lastmod ignore-reload reload-into-ims
> ignore-no-cache reload-into-ims ignore-no-store
> ignore-must-revalidate
>
> logformat combined %>a '-' '-' [%tl] "%rm %ru %>Hs HTTP/%rv"
> "%{User-Agent}>h" %>st %<st %Ss
>
> access_log /var/log/squid/access.log combined
> cache_log /var/log/squid/cache.log
> cache_store_log /var/log/squid/store.log
> buffered_logs on
> strip_query_terms
>
> I tried acl's but no result either:
>
> acl GET method GET
> acl POST method POST
>
> acl FM_url url_regex ^http//api.filemobile.com/services/json/
> cache allow POST FM_url
> cache allow GET FM_url

 This might have worked to evade your "deny JS" rule. If you had left
 off the final '/' after '/json'.

 Note how the URL has "/services/json?" instead of "/services/json/?"

 I would alter the FM_url ACL definition and write the earlier "no_cache
 deny JS" line as: cache deny JS !FM_url

 Amos
Received on Wed Nov 16 2011 - 01:16:52 MST