Ellipsis

I love it when 82 characters do a ton of work for me.

Today I’m trying to debug Courier-IMAP ssl errors that are reported in the log file as “DEBUG: Unexpected SSL connection shutdown”. We’re using QMail so the times in the log file are in tai64, and I’m using Eric Rescorla’s SSLDump:”http://www.rtfm.com/ssldump/” to debug the SSL traffic, which reports time in epoch seconds. I’d like to correlate the SSL traffic with the logged errors. Here’s how:

In window 1, I monitor the QMail logs with:

tail -f /var/log/qmail/imap4-ssl/current | tail64nlocal

which gives output like:

2007-10-30 06:26:02.322254500 tcpserver: status: 30/40
2007-10-30 06:26:02.322255500 tcpserver: pid 9635 from 1.2.3.4
2007-10-30 06:26:02.322257500 tcpserver: ok 9635 buzz.example.net:10.1.1.20:993 :216.220.209.17::57693
2007-10-30 06:26:02.439369500 DEBUG: Connection, ip=[1.2.3.4]

In window 2, I monitor the SSL traffic with:

sudo ssldump -e -k imap.example.com.pem  port 993 | ~/bin/epoch2local.rb

where the script @epoch2local.rb is:

#!/usr/bin/ruby -p

$_.sub!(/1\d{9}/) { |t| Time.at(t.to_f).strftime("%Y-%m-%d %H:%M:%S") }

A quick dissection:

• ruby -p places your code in a while gest; ....; print; end loop
• $_ is the current line
• .sub! does in place substitution, changing the value of $_
• sub!(pattern) { |match| block } the string matting pattern is passed into the {} block as the variable match. The result from the block is substituted for the original string
• /1\d{9}/: assume that a 10-digit number starting with 1 is the epoch time (true for another 30 years or so)
• |t| Time.at(t.to_f).strftime("%Y-%m-%d %H:%M:%S") : PFM. No, not really. Pass the match into the block as t. Convert to a float, then a Time value, then format as %Y-%m-%d %H:%M:%S

I then get SSLDump out put as:

23 21 2007-10-30 06:41:19.1945 (0.1309)  C>S  application_data
24 3  2007-10-30 06:41:19.2474 (0.1185)  S>C  Handshake
      Certificate
24 4  2007-10-30 06:41:19.2474 (0.0000)  S>C  Handshake
      ServerHelloDone  

And I can match up the times with the QMail logs. Fini (although I still have the original question to resolve)

Here’s a noobie mistake. A daemon I have running to report on new files being uploaded to a webserver started dying on me when the filenames had a ’%’ in them.

I was doing a complete ‘Duh!’ coding mistake. Take this program:

#!/usr/bin/ruby  -w
require 'syslog'

PROGRAM_NAME="testlog" 
LOG_FACILITY=Syslog::LOG_LOCAL2

$log=Syslog.open(PROGRAM_NAME, Syslog::LOG_PID, LOG_FACILITY)
$log.info("Starting args: " + ARGV.join(" "))

exit

If you run it:

t.rb my message

You’ll get this in the log file:

Oct 23 08:14:06 raymond testlog[7570]: Starting args: my message

However try this:

$ .rb my "%message" 

./t.rb:11:in `info': malformed format string - %m (ArgumentError)
        from ./t.rb:11

The problem is that syslog interprets ’%’ in the message string as a printf style format character. That’s the way of the underlying Unix library, like it or not. And the code will barf if you try @$log.info(“Starting args #{variable}”). The correct way to code is this:

$log.info("Starting args: %s", ARGV.join(" "))

and the ’%s’ gets the argument string value substituted in.

One could write here about the need to sanitize tainted input, but I won’t.

I’m finally plugging away at at my first wholly-owned Rails application. It be late, so I’ll just briefly note that one can do SQL LIKE statements with MySQL semi-safely with the following snippet:

find(  :all, 
         :conditions => ["description RLIKE ?", '.*' + description + '.*'] )

while gets translated into the following SQL:

SELECT * FROM waypoints WHERE (description RLIKE '.*Illinois.*') 

I’ve not yet read the security chapter for AWDwR, so I don’t know whether this is recommended or not

Note to self: Shouldn’t this be on the Rails Wiki?

_

Also, I keep getting caught with ‘You have a nil object when you didn’t expect it! You might have expected an instance of Array’, even when I was checking if an object was Nil or not. Now I’ve learned that checks like this won’t work:

if params[:waypoint][:stop].nil?
     @waypoints = []
end

because they were evaluating to something like nil[:stop].nil?, which don’t fly. I now have code that does this, but it’s ugly, so I need to ask around about what the proper way to do this is.

if params.nil? || params[:waypoint].nil?
     @waypoints = []
end

Stefan Saasen points out that the Ruby MySQL bindings are broken on Intel Macs.

Here’s my version of the fix:

sudo gem install mysql -- --include=/opt/local
# select option 2 for Ruby 2.7, it will fail to build
cd /opt/local/lib/ruby/gems/1.8/gems/mysql-2.7
sudo ruby extconf.rb install mysql -- --with-mysql-dir=/opt/local
vim mysql.c
# now here's where you add the line '#define ulong unsigned long' 
# just before the line '#define MYSQL_RUBY_VERSION...'
sudo make
sudo make install

Yesterday I installed Ruby on Rails on my new(-ish) Intel MacBook. Last time around I built with a combination of Fink packages and hand-built applications following this posting at Hivelogic

This time around I’ve been using MacPorts, and it’s making my life much easier. Evan Weaver got me started with his post on building ruby, rails and associated pieces, but enough has changed changed since June 26 to merit my own updated take on the process.

Getting started

As Evan notes, “First, install the Apple Xcode tools from your OS X installation disc”. Please do so.

Next, install a recent version of “MacPorts” (what used to be known as DarwinPorts) from their Subversion respository. Installing from a .dmg file is easiest, then you can let MacPorts upgrade itself later on. As of this writing, Ports 1.3.2 is out, but disk images are only available for 1.3.1, e.g. at DarwinPorts-1.3.1-10.4.dmg

PATH="/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin" 

Install the ports

Now open a terminal (/Applications/Utilities/Terminal) and run the following:

sudo port -d selfupdate 
sudo port install lighttpd +ssl 
sudo port install rb-rubygems
sudo port install rb-fcgi
sudo port install mysql4 +server

Set up MySQL

You’ll also need to get mysql4 set up with these commands:

# set up the mysql database:
sudo -u mysql mysql_install_db
# start the server:
sudo /opt/local/bin/mysqld_safe --user=mysql
# set the root password (picking your own password, of course)
/opt/local/bin/mysqladmin -u root password newpassword

If you want Launcher to start MySQL automatically on reboot, you can run the following:

sudo launchclt load -w  \
  /Library/LaunchDaemons/org.macports.mysql4.plist
# stop the server
sudo launchctl stop org.macports.mysql4
# start the server
sudo launchctl start org.macports.mysql4

Install the gems

Running gems with the ‘-y’ option automatically takes care of prerequisites

sudo gem install -y rails
sudo gem install -y capistrano

Test!

First, are you hitting the right version of Ruby? ruby --version should return something like ruby 1.8.5 (2006-08-25) [i686-darwin8.8.1] not this: ruby 1.8.2 (2004-12-25) [universal-darwin8.0]

Next, can you build a Rails application with

cd ~/tmp
rails widgetapp

Okay? Good. Now let’s cd widgetapp and put the database through it’s paces. Save the following code as test_rails_db.sh (or download it here)

#!/bin/sh

echo -n "Enter MySQL root password: " 
read PASSWD

mysqladmin -u root -p$PASSWD create widgetapp_development

cat >db/create.sql <<EOF
DROP table if exists widgets;

CREATE table widgets (
    id  int not null    auto_increment,
    name    varchar(40)  not null,
    description   varchar(100)   not null,
    primary key (id)
    );

INSERT INTO widgets (name, description) VALUES ("Tool", "Useful item");
INSERT INTO widgets (name, description) VALUES ("Food", "Tasty stuff");

EOF

mysql -u root -p$PASSWD  widgetapp_development < db/create.sql

mv config/database.yml config/database.yml.dist

cat >config/database.yml <<EOF 

development:
  adapter: mysql
  database: widgetapp_development
  username: root
  password: $PASSWD
  socket: /opt/local/var/run/mysqld/mysqld.sock

EOF

and run sh ./test_rails_db.sh. Enter your password when prompted.

Now the proof is in the pudding. If the following run s while you’re in your widgetapp rails directory, you’re golden:

script/generate scaffold Widget
script/server

Now browse to http://0.0.0.0:3000/widgets/list and you should utter a little gasp of joy.

Today I interviewed for a position with University of Maryland Office of Information Technology and the University’s College of Chemical and Life Sciences. As part of the interview process I was able to do a short presentation, and I chose to provide an overview of the web content management systems (wikis, blogs, cms’s and backpack) that I’ve worked with, and the role I think they can play in a univeristy environment beyond the strictly instructional.

Some of the resources I referenced in the talk are:

• Wikipedia (of course)
• the wikis hosted at swiki.dlese.org
• RealClimate.org, an excellent blog on climate science
• Buzz, the science blog and the Science Museum of Minnesota
• BZST, a blog by UMd Professor Galit Shmueli
• Plone, the Zope-based CMS
• the Plone sites hosted at www.dlese.org
• the Joomla sites at the College of Chemistry and Life Sciences, the Graduate Students’ Organization and the Doyle Research Group
• 37Signals, the web app development group responsible for Ruby on Rails and Getting Real
• Backpack the place to put all your stuff, also from “37Signals”

Lastly, one can download the PDF version of my presentation

My life is exciting enough that I could spend Friday evening working on my resurrected site.

I’d intended to document the experience well enough that I could update the various wikis on Typo+Dreamhost, but it got so late that my brain was too fried to do anything but randomly edit previous commands in the hope that something would work. I love to tinker around at night, but my brain stalls out a lot earlier than it used to.

The main sticking points I ran into were:

• The SVN checkout of the Rails1.1 typ version took about two bowls of cereal to complete
• The rake migration calls do nothing—I never timed them but after about 5 minutes I start hitting Ctlr-C. I didn’t dig into the strace of rake too deeply, but just set up the databases from within the mysql client with ‘source db/schema.mysql.sql’
• I was getting mysql server not found errors—because I had a TAB character after my hostname in my database.yml file. Why are computers so damn literal?
• I was getting 500 server errors, so I applied the RailsFCGIHandler tweak from Alex Young’s Blog

Of these, only the rake problems merit documenting on a Wiki—and I’ve done so at Typo Trac Wiki