Re: [squid-users] looking for testers: google maps/earth/youtube caching

From: Andreas Pettersson <andpet@dont-contact.us>
Date: Tue, 27 Nov 2007 13:47:11 +0100

I've done some testing with Google Maps using the following rewriting
regex and it seems to work just fine. Lot's of HITs.

  if (m/http:\/\/kh[0-3]?\.google\.com[\.a-z]*\/(.*?) /) {
    print "http://keyhole-srv.google.com.SQUIDINTERNAL/" . $1 . "\n";

However with YouTube I have some trouble. This is the rewriter I use:

  } elsif
(m/^http:\/\/\d+\.\d+\.\d+\.\d+\/get_video\?video_id=(.*?)&origin=.*?youtube\.com
/) {
    # Zlatan skojar med Ekwall
    #
http://74.125.10.100/get_video?video_id=hDJ7IRS4krA&origin=nyc-v23.nyc.youtube.com
    print
"http://video-srv.youtube.com.SQUIDINTERNAL/get_video?video_id=" . $1 .
"\n";

When testing outside squid it looks ok.

# echo
"http://74.125.10.100/get_video?video_id=hDJ7IRS4krA&origin=nyc-v23.nyc.youtube.com
" | ./storeurl_rewrite.pl
http://video-srv.youtube.com.SQUIDINTERNAL/get_video?video_id=hDJ7IRS4krA

The first fetch is of course a TCP_MISS and the debug log gives me this:

2007/11/27 13:18:07| storeurlStart:
'http://74.125.10.100/get_video?video_id=hDJ7IRS4krA&origin=nyc-v23.nyc.youtube.com'
2007/11/27 13:18:07| storeurlStart: sending
'http://74.125.10.100/get_video?video_id=hDJ7IRS4krA&origin=nyc-v23.nyc.youtube.com
127.0.0.1/localhost - GET - myip=127.0.0.1 myport=8081' to the helper
2007/11/27 13:18:08| storeurlHandleRead:
{http://video-srv.youtube.com.SQUIDINTERNAL/get_video?video_id=hDJ7IRS4krA}

Subsequent fetches (squidclient and wget) using the exact same URL are
TCP_HITs.
But. If I for example change the origin part of the URL (v23 to v24):

2007/11/27 13:24:05| storeurlStart:
'http://74.125.10.100/get_video?video_id=hDJ7IRS4krA&origin=nyc-v24.nyc.youtube.com'
2007/11/27 13:24:05| storeurlStart: sending
'http://74.125.10.100/get_video?video_id=hDJ7IRS4krA&origin=nyc-v24.nyc.youtube.com
127.0.0.1/localhost - GET - myip=127.0.0.1 myport=8081' to the helper
2007/11/27 13:24:05| storeurlHandleRead:
{http://video-srv.youtube.com.SQUIDINTERNAL/get_video?video_id=hDJ7IRS4krA}

As you see, the resulting storeURL is the same as before, but the
request is now a TCP_SWAPFAIL_MISS.
Shouldn't this be a HIT as well? Something I've missed?

-- 
Andreas
Received on Tue Nov 27 2007 - 05:49:13 MST

This archive was generated by hypermail pre-2.1.9 : Sat Dec 01 2007 - 12:00:02 MST