Apache and "shared memory" issues on Linux

Peter Burkholder — Thu, 10 Apr 2008 06:07:00 -0700

This morning I was having trouble getting Apache (2.0.X) to restart. I was getting these error messages:

[Thu Apr 10 08:32:49 2008] [crit] (17)File exists: unable to create scoreboard "/var/private/logs/apache_runtime_status" (name-based shared memory failure)

and I kept deleting the offending file, and making sure that permissions all along the path were correct. No dice.

So I restarted Apache again running strace:

strace /usr/sbin/httpd2-prefork -X -Dprivate '-CPidFile /var/run/apache2.private.pid' -f /etc/apache2/httpd.conf

and saw output like this:

unlink("/var/private/logs/apache_runtime_status") = -1 ENOENT (No such file or directory)
open("/var/private/logs/apache_runtime_status", O_WRONLY|O_CREAT|O_EXCL, 0666) = 9
stat("/var/private/logs/apache_runtime_status", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
shmget(16908303, 40824, IPC_CREAT|IPC_EXCL|0600) = -1 EEXIST (File exists)
write(2, "[Thu Apr 10 08:18:46 2008] [crit"..., 168) = 168

Red Herring

Turns out that worrying about the filesystem was a distraction. The real issue is that the previous Apache had left behind shared memory segments that it couldn’t access anymore. Now I won’t pretend that I understand the ins and outs of shared memory on Linux, but a bit of Googling led me, fortunately to Sven Vermeulen’s blog, where he shared a similar experience on Solaris. I would’ve left nice comments on his blog, but as he doesn’t take comments, I’ll give a shout out to him here.

Anyhow, the ipcs and ipcrm commands come to the rescue:

# ipcs -a

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status
0x0102000f 99942402   root      600        40824      0

------ Semaphore Arrays --------
key        semid      owner      perms      nsems

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages
0x00001f58 0          root       600        0            0

Ah-ha—there’s shmid at 99942402. Let’s rm that:

# ipcrm -m 99942402
# ipcs -a

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status

------ Semaphore Arrays --------
key        semid      owner      perms      nsems

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages
0x00001f58 0          root       600        0            0

And apache started right up. Yea.

P.S. One find out more about a shared memory segment with something like: ipcs -m -i 99942402, which will report:

Shared memory Segment shmid=99942402
uid=0   gid=0   cuid=0  cgid=0
mode=0600       access_perms=0600
bytes=40824     lpid=6987       cpid=6987       nattch=0
att_time=Thu Apr 10 08:20:48 2008  
det_time=Thu Apr 10 08:21:55 2008  
change_time=Thu Apr 10 08:20:48 2008

"Apache and "shared memory" issues on Linux" by christt

Mon, 03 Nov 2008 13:23:52 -0800

Thanks Peter! You saved my day!

"Apache and "shared memory" issues on Linux" by rizahnst

Wed, 16 Jul 2008 03:23:32 -0700

thanks!

"Apache and "shared memory" issues on Linux" by Anny

Sun, 20 Apr 2008 00:39:59 -0700

I have wanted one of these forever! THANKS for the great work

Ellipsis: Apache and "shared memory" issues on Linux

Apache and "shared memory" issues on Linux

Red Herring

"Apache and "shared memory" issues on Linux" by christt

"Apache and "shared memory" issues on Linux" by rizahnst

"Apache and "shared memory" issues on Linux" by Anny