Yesterday I rescued a co-worker’s computer from a bad 10.4.9 update. My salvation was this post, to which I added my own comments.

At least I learned a few general principles, namely that mucked up Macs can be saved by the general principle that I carried in from Linux – boot from Knoppix and repair the damage. In this case, we boot from an Install disk (one which explicitly says “Hold down C to to boot to Install” and the Installer will give you a Terminal and a command line. From their it’s a matter of running hdutil and open to mount disk images and launch applications.

I’m having to come up to speed on managing fleet of Macs, and fast, or else their going to eat up all my time. I’d like a week to play with OS X server and Puppet, thank you very much.

I saw this mentioned on the dc-sage email list but missed the particular example, until Sweth Chandramouli posted the following example for testing whether nameservers in /etc/resolv.conf are actually working:

#!/bin/sh
while read TOKEN IP ; do
   case $TOKEN in nameserver )
      echo "Testing DNS query against $IP: `dig -x 127.0.0.1 @$IP | grep ';; ->>'`" |\
      logger -p local3.info -t check_dns ;;
   esac
done < /etc/resolv.conf

Nice trick, although I’d like to cat the input into while at the top:

cat /etc/resolv.conf |
while ...

Oh yes, I have a new job. Director of System and Network Administration at EchoDitto, which is a topic meriting several blog posts.

Yesterday I installed Ruby on Rails on my new(-ish) Intel MacBook. Last time around I built with a combination of Fink packages and hand-built applications following this posting at Hivelogic

This time around I’ve been using MacPorts, and it’s making my life much easier. Evan Weaver got me started with his post on building ruby, rails and associated pieces, but enough has changed changed since June 26 to merit my own updated take on the process.

Getting started

As Evan notes, “First, install the Apple Xcode tools from your OS X installation disc”. Please do so.

Next, install a recent version of “MacPorts” (what used to be known as DarwinPorts) from their Subversion respository. Installing from a .dmg file is easiest, then you can let MacPorts upgrade itself later on. As of this writing, Ports 1.3.2 is out, but disk images are only available for 1.3.1, e.g. at DarwinPorts-1.3.1-10.4.dmg

PATH="/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin" 

Install the ports

Now open a terminal (/Applications/Utilities/Terminal) and run the following:

sudo port -d selfupdate 
sudo port install lighttpd +ssl 
sudo port install rb-rubygems
sudo port install rb-fcgi
sudo port install mysql4 +server

Set up MySQL

You’ll also need to get mysql4 set up with these commands:

# set up the mysql database:
sudo -u mysql mysql_install_db
# start the server:
sudo /opt/local/bin/mysqld_safe --user=mysql
# set the root password (picking your own password, of course)
/opt/local/bin/mysqladmin -u root password newpassword

If you want Launcher to start MySQL automatically on reboot, you can run the following:

sudo launchclt load -w  \
  /Library/LaunchDaemons/org.macports.mysql4.plist
# stop the server
sudo launchctl stop org.macports.mysql4
# start the server
sudo launchctl start org.macports.mysql4

Install the gems

Running gems with the ‘-y’ option automatically takes care of prerequisites

sudo gem install -y rails
sudo gem install -y capistrano

Test!

First, are you hitting the right version of Ruby? ruby --version should return something like ruby 1.8.5 (2006-08-25) [i686-darwin8.8.1] not this: ruby 1.8.2 (2004-12-25) [universal-darwin8.0]

Next, can you build a Rails application with

cd ~/tmp
rails widgetapp

Okay? Good. Now let’s cd widgetapp and put the database through it’s paces. Save the following code as test_rails_db.sh (or download it here)

#!/bin/sh

echo -n "Enter MySQL root password: " 
read PASSWD

mysqladmin -u root -p$PASSWD create widgetapp_development

cat >db/create.sql <<EOF
DROP table if exists widgets;

CREATE table widgets (
    id  int not null    auto_increment,
    name    varchar(40)  not null,
    description   varchar(100)   not null,
    primary key (id)
    );

INSERT INTO widgets (name, description) VALUES ("Tool", "Useful item");
INSERT INTO widgets (name, description) VALUES ("Food", "Tasty stuff");

EOF

mysql -u root -p$PASSWD  widgetapp_development < db/create.sql

mv config/database.yml config/database.yml.dist

cat >config/database.yml <<EOF 

development:
  adapter: mysql
  database: widgetapp_development
  username: root
  password: $PASSWD
  socket: /opt/local/var/run/mysqld/mysqld.sock

EOF

and run sh ./test_rails_db.sh. Enter your password when prompted.

Now the proof is in the pudding. If the following run s while you’re in your widgetapp rails directory, you’re golden:

script/generate scaffold Widget
script/server

Now browse to http://0.0.0.0:3000/widgets/list and you should utter a little gasp of joy.

Today I interviewed for a position with University of Maryland Office of Information Technology and the University’s College of Chemical and Life Sciences. As part of the interview process I was able to do a short presentation, and I chose to provide an overview of the web content management systems (wikis, blogs, cms’s and backpack) that I’ve worked with, and the role I think they can play in a univeristy environment beyond the strictly instructional.

Some of the resources I referenced in the talk are:

• Wikipedia (of course)
• the wikis hosted at swiki.dlese.org
• RealClimate.org, an excellent blog on climate science
• Buzz, the science blog and the Science Museum of Minnesota
• BZST, a blog by UMd Professor Galit Shmueli
• Plone, the Zope-based CMS
• the Plone sites hosted at www.dlese.org
• the Joomla sites at the College of Chemistry and Life Sciences, the Graduate Students’ Organization and the Doyle Research Group
• 37Signals, the web app development group responsible for Ruby on Rails and Getting Real
• Backpack the place to put all your stuff, also from “37Signals”

Lastly, one can download the PDF version of my presentation

This is also posted on MacOSX Hints

With Mac OS X Tiger (10.4.8, to be exact), Mail.app 2 (2.1.1) and Thunderbird 1.5 (1.5.0.7) the import into Mail.app is very straightforward for importing the Local Folders and Inbox from a Thunderbird POP account. I did this as two imports, first the Local Folders, then the INBOX

Before doing anything, use Thunderbird to “Compact Folders” (which removes messages that have been marked as deleted in your folders).

1) in Mail, select File -> Import Mailboxes, then select Other 2) select the folder (HOME)/Library/Thunderbird/Profiles/Default.lay/Mail/Local Folder 3) Mail then finds all the importable mbox files therein, and all the nested folders and files as well. Confirm the import

Then repeat, but using (HOME)/Library/Thunderbird/Profiles/Default.lay/Mail/(account name) as the base for the import

This leaves you with two new top-level folders in Mail.app: Import and Import-2. You can then rename and re-arrange as needed for your purposes

I’ve migrated a lot of my old sysadmin stuff from my old host. This includes the following stuff

Presentations

• UCAR Web Advisory Group Shibboleth Presentation, October 2005
• UCAR Content Management System Forum, October 2004
• Web Application Security. A presentation for the Denver chapter of ISSA, 10 April 2002.
• SSH & SSL for SysAdmins. A presentation for University of Washington NetSys/LanAdmin, 24 January 2002.
• SSH & SSL. A presentation for the Colorado chapter of SAGE (Co-SAGE)”, 14 Nov 2001.

Notes

Some notes on work I do, which may or may not be useful to others

• How to run your own CA for launching SSL attacks
• PortSentry
• VPNs using Free S/WAN

Articles

• SSL Man-in-the-Middle Attacks 1 Feb 2002. An evaluation of several SSL attacks, includes a modification to Dug Song’s webmitm exploit.

RSS Scraper Job Feeds

Since I’m looking for work with specific institutions of interest in the Washington, DC, area, I thought I could track jobs more easily if I could subscribe to the relevant RSS feeds. Alas, although any institution now has its “Job Opportunities” web page, they seldom have RSS feeds of those pages. So I decided to work out my Ruby and Regular Expression skills and build my own feeds. These are built from RubyForge RSSscraper, written by Monster or <a href=”http://www.dice.com”>Dice or their ilk, so I can’t compare this to what they offer. But I find it useful so far, although scraping HTML from the NSF has been really challenging. I’d recommend that one sign up with their email job notification service.

RSS Job Feeds

• AAAS Jobs
• NSF Jobs
• Univ. of Maryland, College Park, Jobs

I haven’t used Expect in years, for some reason, but yesterday I need a Makefile that would, as part of a build process, checkout some project using anonymous CVS. Here’s the solution:

#Makefile
CVSROOT         = :pserver:anonymous@strandmapservic.cvs.sourceforge.net:/cvsroot/strandmapservic

BRANCH = HEAD
PROJECTS = smsdemo

define CVSlogin
  /usr/bin/expect -c 'spawn cvs login' -c 'expect "CVS password:"' -c 'send "\r"' -c 'expect eof'
endef

#...
checkout: 
        echo $(CVSROOT) | if grep -q pserver; then \
                 $(CVSlogin); \
            fi
        CVSROOT=$(CVSROOT) cvs co -r $(BRANCH) $(PROJECTS)
        touch $@

If some of this seems overly complicated, it’s because this same Makefile (or rather, the Makefile.in that’s processed by m4) is used for many other build.

echo :pserver:anonymous@strandmapservic.cvs.sourceforge.net:/cvsroot/strandmapservic | if grep -q pserver; then \
           /usr/bin/expect -c 'spawn cvs login' -c 'expect "CVS password:"' -c 'send "\r"' -c 'expect eof'; \
    fi
spawn cvs login
Logging in to :pserver:anonymous@strandmapservic.cvs.sourceforge.net:2401/cvsroot/strandmapservic
CVS password: 
CVSROOT=:pserver:anonymous@strandmapservic.cvs.sourceforge.net:/cvsroot/strandmapservic cvs co -r HEAD smsdemo
cvs checkout: Updating smsdemo
U smsdemo/.classpath
...

From SANS:

Colleges Struggle with Cyber Security (2 & 1 August 2006) Cyber security breaches at colleges and universities accounted for one-third to one-half of all reported cyber security incidents in the last 18 months. This could be attributed in part to the fact that schools are likely to be more forthcoming about breaches than are private sector organizations. In addition, computer systems at institutions of higher education are often decentralized, making security more difficult. This series of articles examines “how and why security breaches have occurred” at the schools and looks at the increased privacy and security measures colleges and universities have taken in response to the cyber security incidents.

College Hacking #1

College Hacking #2

College Hacking #3

[Editor’s Note (Kreitner): Establishment and enforcement of security is a widely neglected management responsibility. One of management’s duties is assigning accountability for adherence to established policies and articulating consequences for policy violations. Management’s failure to do this for protecting information is a root cause of most security incidents. Once Boards of Directors begin to routinely require management to report to the Board on every security incident, including its cause and what has been done to prevent a similar incident in the future, management will begin to get serious about protecting the information entrusted to the organization.]

A testbed for procmail

What I’d like is a way to test my production set of procmail rules without procmail actually sending any email, or triggering any programs. That way any mistakes in my rules would not end up, say, paging my boss or flooding our trouble ticket system. I didn’t see any—dry-run support in procmail, so I decided to tackle the problem with m4.

To keep the example simple, instead of our usual procmailrc file we’ll use one that has the action parts of the recipe, like ’| ’ or ’!’ replaced with tokens that will later be processed by m4. E.g.:

    include(`procmail.m4')dnl

    :0 
    * ^TO_pburkholder
    __FILEIN /tmp/file    

    :0 
    * ^Subject: nothing
    __SENDTO root@localhost         #

    :0 
    * ^From:.*root
    __PIPETO /bin/rm some_file      #

    :0
    __FILEIN /tmp/pburkholder

and a procmail.m4 file as follows:

divert(-1)dnl
ifdef(`LIVE',`
        define(`__LOGFILE', `/tmp/logfile')
        define(`__FILEIN', `')
        define(`__SENDTO', `!')
        define(`__PIPETO', `|')
        ')

ifdef(`TEST',`
        define(`__LOGFILE', `')
        define(`__FILEIN', `LOG=| echo file into ')
        define(`__SENDTO', `LOG=| echo send to ')
        define(`__PIPETO', `LOG=| echo pipe to ')
        ')
divert(0)dnl

test: 
        m4 -D TEST  procmail.in > procmailrc
        @for i in testmsg/*; do \
            echo Testing with message: $$i; \
            procmail ./procmailrc < $$i; echo ;\
        done

live:
        m4 -D LIVE procmail.in > .procmailrc

Then a run of make test produces this output:

m4 -D TEST  procmail.in > procmailrc
Testing with message: testmsg/empty_message
file into /tmp/peterb
Testing with message: testmsg/subject_nothing
send to root@localhostfile into /tmp/peterb
Testing with message: testmsg/to_peterb
file into /tmp/filefile into /tmp/peterb

by writing a procmail file that looks like this:

:0 
* ^TO_peterb
LOG=| echo file into  /tmp/file 
...

where the ‘LOG=’ action will cause procmail to echo the subsequent text to stderror. The ‘make live’ call will produce the real .procmailrc, which looks like this:

:0 
* ^TO_peterb
 /tmp/file
...

The beauty of this is that my test case is not divorced from the production case—they use exactly the same input file. All I need to do is build up a representative folder full of test messages. If I wanted to get really fancy I could put a Perl or Ruby wrapper around the test calls and only squawk if the desired output was not seen for any test case.

If I were feeling clever, I’d see if I could replace m4 with ERB, and Make with Rake, but this is part of my m4 m4ania series, so what would be the point.

Acknowledgements

Timo’s procmail tips

• M4 and procmail testing
• M4 and httpd.conf maintenance
• M4 and a complex build system for multiple environments

I think m4 has a lot to offer system administrators and web developers who need to maintain configuration files in a manner that are beyond the scope of simple sed, awk or cfenging scripts.

More to come on this topic