Apache redirects with RewriteMap and RewriteCond 47
At work we’re migrating several thousand research articles from Zope to another CMS. The CMS folks are taking care of moving the content, but when we’re done we’re going to institute a boatload of R=301 redirects from the old URLs to the new URLs.
RewriteMap is the accepted way with mod_rewrite of handling a lot of one-to-one mappings that don’t follow any particular pattern. What I figured out today was that I can use a rewrite map in a RewriteCond statement so I only do the redirect when there’s a match in the rewrite map lookup.
Here are some snippets from my httpd.conf to illustrate:
For testing, we’ll want logging:
RewriteLog /var/log/httpd/rewrite.log
RewriteLogLevel 2
Define the map we’re using. The content of the map is ‘old_uri new_uri’ with a space separating the two. Use .txt for testing, and the below we’ll convert to a DBM.
RewriteMap research_map txt:/etc/httpd/conf/research_map.txt
Here, I got a hint from http://www.tunnell.org/blog_posts_view.php?blog_postid=3. Remember that the syntax for a RewriteCond is:
RewriteCond TestString ConditionPattern
TestString will be a map lookup of $1, where $1 is the match string of the
following RewriteRule, expressed: ${research_map:$1}
For ConditionPattern,
we will test if the TestString is lexically greater than
””, the empty string, which is what the map lookup returns when there’s no
match. Expressed: >""
RewriteCond ${research_map:$1} >"" # IE, if map result is greater than ""
So if the URL starts with /research, then
use the research_map value for the key $1 to redirect to new address
RewriteRule ^(/research/.*$) ${research_map:$1} [R=301,L]
What we end up with is a few lines of configuration that quickly let me put in place 3342 new redirects. Here’s the whole stanza:
RewriteMap research_map txt:/etc/httpd/conf/research_map.txt
RewriteCond ${research_map:$1} >"" # IE, if map result is greater than ""
RewriteRule ^(/research/.*$) ${research_map:$1} [R=301,L]
# Same thing, but lookup with a trailing slash if there isn't one
RewriteCond ${research_map:$1/} >""
RewriteRule ^(/research/.*[^/]$) ${research_map:$1/} [R=301,L]
Lastly, converting the textfile to a dbm speeds up the lookup by at least an order of magnitude.
Apache and "shared memory" issues on Linux 5
This morning I was having trouble getting Apache (2.0.X) to restart. I was getting these error messages:
[Thu Apr 10 08:32:49 2008] [crit] (17)File exists: unable to create scoreboard "/var/private/logs/apache_runtime_status" (name-based shared memory failure)
and I kept deleting the offending file, and making sure that permissions all along the path were correct. No dice.
So I restarted Apache again running strace:
strace /usr/sbin/httpd2-prefork -X -Dprivate '-CPidFile /var/run/apache2.private.pid' -f /etc/apache2/httpd.conf
and saw output like this:
unlink("/var/private/logs/apache_runtime_status") = -1 ENOENT (No such file or directory)
open("/var/private/logs/apache_runtime_status", O_WRONLY|O_CREAT|O_EXCL, 0666) = 9
stat("/var/private/logs/apache_runtime_status", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
shmget(16908303, 40824, IPC_CREAT|IPC_EXCL|0600) = -1 EEXIST (File exists)
write(2, "[Thu Apr 10 08:18:46 2008] [crit"..., 168) = 168
Red Herring
Turns out that worrying about the filesystem was a distraction. The real issue is that the previous Apache had left behind shared memory segments that it couldn’t access anymore. Now I won’t pretend that I understand the ins and outs of shared memory on Linux, but a bit of Googling led me, fortunately to Sven Vermeulen’s blog, where he shared a similar experience on Solaris. I would’ve left nice comments on his blog, but as he doesn’t take comments, I’ll give a shout out to him here.
Anyhow, the ipcs and ipcrm commands come to the rescue:
# ipcs -a
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x0102000f 99942402 root 600 40824 0
------ Semaphore Arrays --------
key semid owner perms nsems
------ Message Queues --------
key msqid owner perms used-bytes messages
0x00001f58 0 root 600 0 0
Ah-ha—there’s shmid at 99942402. Let’s rm that:
# ipcrm -m 99942402
# ipcs -a
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
------ Semaphore Arrays --------
key semid owner perms nsems
------ Message Queues --------
key msqid owner perms used-bytes messages
0x00001f58 0 root 600 0 0
And apache started right up. Yea.
P.S. One find out more about a shared memory segment with something like: ipcs -m -i 99942402, which will report:
Shared memory Segment shmid=99942402
uid=0 gid=0 cuid=0 cgid=0
mode=0600 access_perms=0600
bytes=40824 lpid=6987 cpid=6987 nattch=0
att_time=Thu Apr 10 08:20:48 2008
det_time=Thu Apr 10 08:21:55 2008
change_time=Thu Apr 10 08:20:48 2008