Re: Squid regex fast purge.

On Thu, 5 May 2005, Ernest Rider wrote:

> Recently a requirement came up to be able to fast purge squid entries by
> URL matching.

Quite common request. Unfortunately not very easy to implement due to the
design of the cache. The URL is for the major part of the cache only kept
on disk, not in memory.

> void
> storeRemoveRegex(const char *regex) {
> hash_link *walker;
> int i=0;
> regex_t compare;
> regmatch_t pmatch;
> debug(20, 2) ("storeRemoveRegex: Constructing Regex with %s\n",regex);
> int code = regcomp(&compare, regex, REG_EXTENDED | REG_ICASE | REG_NEWLINE);
> if(!code) {
> debug(20, 2) ("storeRemoveRegex: Constructing Regex Done\n");
> debug(20, 2) ("storeRemoveRegex: Finding first entry\n");
> hash_first(store_table);
> debug(20, 2) ("storeRemoveRegex: Finding first entry done\n");
> debug(20, 2) ("storeRemoveRegex: Finding initial entry\n");
> walker=hash_next(store_table);
> debug(20, 2) ("storeRemoveRegex: Finding initial entry done (%p)\n",walker);
> debug(20, 2) ("storeRemoveRegex: Entering Loop\n");
> while(walker) {
> debug(20, 2) ("storeRemoveRegex: Start Loop block with %p\n",walker);
> StoreEntry *store_entry = (StoreEntry *) walker;
> if(store_entry->mem_status == NOT_IN_MEMORY) {
> debug(20, 2) ("storeRemoveRegex: The Entry object was null doing persistence retrieval\n");
> <WHAT SHOULD GO HERE>

Exacly.. this is the problem... The information you need is not available
in memory and you need to swap in the object from disk.

This is mostly the same as the external purge tool (see related software),
except that doing it within Squid is a little more complex due to the
multiplexed nature of Squid.

> debug(20, 2) ("storeRemoveRegex: The Entry object retrieved has pointer %p\n", store_entry->mem_obj);
> }
> debug(20, 2) ("storeRemoveRegex: Executing regex match on %s",storeUrl(store_entry));
> if(!regexec(&compare, storeUrl(store_entry),1, &pmatch,0)) {
> storeLockObject(store_entry);
> debug(20, 2) ("storeRemoveRegex: Releasing Item (%5d) from the store:\n",i);
> storeEntryDump(store_entry,2);
> storeRelease(store_entry);

This part is fine. As Squid is singlethreaded there is no locking needed.

The loop may hit the same entry more than once (twice, once on the public
location, and then once again on the private key location) if it is in
use, but it is safe to call storeRelease multiple times so this is not a
problem.

Be warned that the hash_next function is not safe to be used in event
operations. There can only be one hash_next iteration at a time. Some
juggling will be required for doing this in a reasonable manner in an
incremental event function without running into locking/concurrency
issues.

Regards
Henrik
Received on Fri May 06 2005 - 14:49:08 MDT