Re: Data all piles up in one directory.

On Wed, Jan 12, 2000 at 02:13:13PM +1100, Thorne Lawler wrote:
> Newbie question:
>
> Given a Squid 2.2 system set up, essentially, according to the Quickstart
> instructions, with only one significant modification; I have increased the
> 'maximum_object_size' to '40 MB'; why would all the cache-data e being
> placed in a single cache-directory?

...
> --------------------------
> When Squid wants to create a new disk file for storing an object, it first
> selects which cache_dir the object will go into. This is done with the
> storeDirSelectSwapDir() function. If you have N cache directories, the
> function identifies the 3N/4 (75%) of them with the most available space.
> These directories are then used, in order of having the most available
> space. When Squid has stored one URL to each of the 3N/4 cache_dir's, the
> process repeats and storeDirSelectSwapDir() finds a new set of 3N/4 cache
> directories with the most available space.
> --------------------------

cache_dir here has a specialized meaning, not the same as UNIX
directory. In general terms it would better be described as cache
directory-hierarchy, or in most cases (where Squid is given a chunk of
disk to itself) cache partition or cache file-system. Directories
within that cache partition are filled "from the bottom up", filling
cache_dir/00/00 until 256 files have been stored there, then moving to
cache_dir/00/01, etc.

 
> Given that this is the case, after ~2 weeks of operation under moderate
> load, with a total of 450MB of space, and a cache_dir line which reads:
>
> --------------------------
> cache_dir /squid/cache 400 16 256
> --------------------------

This is a single cache_dir (note the keyword used to define it is the
same as in the explanation above) so everything goes in there, and the
subdirectories in it are filled from the bottom up as you observed.

> The cache also appears to be repacing cache data at what seems(to me) to be
> a very rapid rate... I will probably understand this better as I read more
> of the on-line support documentation.

A significant issue is that 450MB will not hold very much web data,
especially if you defined the largest cacheable object to be 40MB. Ten
users downloading different pieces of demo software, MP3s, or quicktime
movies off the web will completely empty your cache. Another way of
looking at it is that 450MB is about 41 minutes of full-throttle T1
traffic - not long! If that's all you can allocate to the cache, I'd
suggest turning the maximum object size way way down. Try adding more
disk, or setting the maximum object size to 8KB, where at least it
should end up with a lot of cached button GIFs/etc. from popular sites.

A good hit rate is around 30%; I doubt you can get near that with your
current configuration, because objects will get flushed out of the
cache before Squid would even have a chance to figure out if they're
popular or not.
 
> Is this a known behaviour?

Yep to both.

  -- Clifton

-- 
 Clifton Royston  --  LavaNet Systems Architect --  cliftonr@lava.net
        "An absolute monarch would be absolutely wise and good.  
           But no man is strong enough to have no interest.  
             Therefore the best king would be Pure Chance.  
              It is Pure Chance that rules the Universe; 
          therefore, and only therefore, life is good." - AC