Re: squid-1.2-SQUIDFS

--MimeMultipartBoundary
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Sat, 30 May 1998, Stephen R. van den Berg wrote:

> I fail to see why this should be the natural conclusion.
> mmap()ing files basically *always* saves memory and therefore reduces
> net disk activity (which is what it is all about, we don't care if we
> page, or swap, or read/write files; all we care about is that we want
> to minimise disk activity, and minimise the total memory required).

Here is the "worst case" scenario to illustrate Henrik's point (Squid
process must not page):

Assume that the entire disk cache is mmap()-ed. Assume, you have almost
enough memory, but you are short on 100K only. Assume the OS decides to swap
out main.o portion of the Squid process. This would require, say, 10 disk
I/Os. The net disk activity is, thus, minimized (no I/Os after everything is
prefetched into memory). However, since main.o is on disk, Squid does nothing
but waiting for being paged in.

Of course, this is a hypothetical, worst case example. However, I do not
believe in "super smart OSes" that will know exactly what to swap and what to
preserve. Moreover, IMO, the more tasks you hand over to OS, the better the
chances that it will make suboptimal (for your application) decision are. As
far as I understand, none of the Unix OSes was _designed_ to handle massive
amounts of mmap()-ed files (by count and/or volume). Thus, I would expect a
lot of "surprises" when it comes to real world performance.

Clearly, Squid has its own performance problems, but we have more control on
what Squid does compared to what we think an OS might do.

Finally, as far as memory copying is concerned, has anybody calculated the
number of malloc()/memcpy()/strlen()/free() and similar operations Squid does
to process a single request? I would expect that the total overhead Squid
introduces dominates whatever copying is done by the kernel. If somebody
cares about this overhead, the place to start is Squid, not the OS, I guess.

$0.02,
 
Alex.

--MimeMultipartBoundary--
Received on Tue Jul 29 2003 - 13:15:50 MDT