Re: squid suggestions

> I recently wrote a perl script that read a word from a text file, opened a
> unique file, wrote the text and then closed that filehandle.

> I found that it was spending at least 20% of it's time in system mode,
> taking forever and being verry ineficient.

    Yes, you are right about the big overhead. Every open has to go
 through lots of OS blabla, including directory searches, inode caches,
 directory caches, buffers, etc. etc. If Squid cache contents is looked
 at like a huge database, then there should be lots of experience to be
 taken from database developers. Many of them suggest to use totally
 raw disks with their own filesystem on them to bypass OS totally. This
 would both reduce memory needs and allows to optimize within application.

> I think that creating one huge file which contains all the cached pages
> could speed up squid substantially.

    There are few problems with that, Having one open file enforces
 serialization of access to it within application, switching off OS-es
 internal optimizations, so to make things fast we'd need to do this
 optimization within squid? Tough task.

    Then, we could use mmap to map the whole file (or any parts of it)
 in virtual memory space, but then we never exactly know when access
 to any part of it will introduce iowait and blocks the whole process
 upon reads. On the other hand, mapped page writes can be made async
 [memcntl(2)], giving OS chance to optimize actual writes and skipping
 write calls with buffer allocations and lots of other file operations.

    Further, using mmap we could lock certain critical memory areas into
 ram, overcoming incredible performance degradation in cases when these
 areas would be otherwise paged out into swap, I think Store Database
 would be the first candidate.

    Using systemwide shared mapped cache contents could allow for using
 multiple instances of squid sharing the same cache - might be useful,
 but also leads coding to the next stage of complexity. But in its
 simpliest form it could be used to allow for very fast squid startups.
 Lets imagine a "dispatcher" proccess running that starts squid and hands
 it over the once mapped in and checked file - no need to every time do
 integrity checks, just attach to it and start serving. Will make crash
 (core) recovery much faster. In its toughest form it might provide kind
 of non-stop fault tolerance and possibly effective use of multiple cpu's
 in the system...

> ---------------------------------
> I think that a lot of development would have to go into the "large file"
> thing... so the extra x<10% time spent less in system time might not
> be worth it.

 Actually, the benefit will get bigger with bigger caches. We here have
 about 0.5M objects in cache, thousands of directories, and I guess that
 substantial part of all disk io is to do with directory lookup and
 inode updates. To make things work faster, I had to give away lots of
 ram for buffering and caching, kernel also takes its needs without
 asking. I have about 30-40MB of ram locked for kernel needs, and thats
 alot!

    On the practical side, using single file will in any case mean total
 rewrite of squid, So, if this will be considered practical, not before
 rewritten squid appears, I guess.
    Using single file we need to implement some efficient virtual file
 system within this file, with its own directory indexes, deal with
 fragmentation, etc. - lots of tricky stuff...
    We will need a special handling for crash recovery, fschk type of
 thingy.
    Select loop cannot go anywhere, so there would be some major headache
 glueing those together in an efficient way. Async IO (kaio) might be very
 useful for reads instead of depending on pagefaults. Btw, socket io might
 read from socket and write directly into mapped file and read directly
 from freshly written parts of it - proxying with cacheing will become
 almost no-memory-hungry deal.
    Squid might use several files for different domains or some other
 criteria, say - use separate databases for local domains, or short-living
 pages, or whatever. If it can be predefined what size each file is, it
 would be much better to utilize different disk sizes...

    In summary, to me it seems to give substantial benefit in terms of
 speed of squid and efficient use of system resources. Then again, this
 means a new squid, alot more complex one.

 Would some of you experienced coders please estimate the level of
 complexity of rewriting squid to use something like this?

-------------------------------------------------------------------
 Andres Kroonmaa Telefon: 6308 909
 Network administrator
 E-mail: andre@ml.ee Phone: (+372) 6308 909
 Organization: MicroLink Online
 EE0001, Estonia, Tallinn, Sakala 19 Fax: (+372) 6308 901
-------------------------------------------------------------------
Received on Tue Jul 29 2003 - 13:15:40 MDT