Debugging Internet Explorer SSL issues with VmWare, IEAutomation and Wireshark 2

Posted by Peter Burkholder Sat, 09 Dec 2006 05:38:00 GMT

This week I happened upon a client who was eager to solve a persistent problem with Microsoft’s Internet Explorer bombing when trying to POST content over HTTPS to a custom web application. The client sent me the thread from the trouble tracking system, and it was clear that they were already aware of the magic Apache mod_ssl incantation to address some of MSIE’s non-compliant behavior. Since they were already barking up that tree, I decided that they needed a better test bed to help confirm whether the problem was truly being addressed by whatever remedies they were hauling out. To put it another way, we couldn’t really apply any scientific method unless we could have a control case and an experimental case.

Since the problem was MSIE specific, I first needed a way to drive MSIE through some test cases and evaluate the results.

Getting a flawed version of MSIE

First, I had to get a sufficiently old version of MSIE, since late versions of IE6 and IE7 are all okay. To do this all safely and reproducibly, I’m running Windows under an instance of VmWare Server on a Linux host. To get things set up, I took care of the following:

(if you have WinXP CD, start there, since IE 6 first came out with Win XP)

That will give you MSIE 6.00.2800.1106, which failed miserably when I ran it through it’s paces to POST content over HTTPS.

Automating MSIE with Perl Win32::IEAutomation

Next, I needed to automate testing with MSIE, and to the rescue comes Prashant Shewale’s Perl module Win32::IEAutomation. To run the module, I did the following on my Win2k System

  • Download and install Microsoft’s nmake.exe. See their Microsoft Knowledge Base, article 132084, and follow link to nmake15.exe. Then run the downloaded file, and move nmake.exe and nmake.err to c:\perl\bin.
  • Run ‘cpan -i Win32::IEAutomation’ from the command line

At this point, I also installed CygWin and some decent editors to so some sane development and testing on the system, but that’s beyond the scope of this article.

Last, I wrote a variant on the following script to drive IE:



    use Win32::IEAutomation;

    # Set up variables
    $server_base="https://www.example.com";
    $wait=$ARGV[0];
    $now=localtime(time);
    $upload="C:\Documents and Settings\Peter Burkholder\My Documents\TextDoc.txt";
    $user="username\@email.com";
    $pass="password";
    $title="PeterB Test for $wait sec at $now";

    # Create new instance of Internet Explorer
    my $ie = Win32::IEAutomation->new( visible => 1, maximize => 1);

    # Goto Login page and Login
    $ie->gotoURL('https://example.com');

    $ie->getTextBox('name:', "username")->SetValue($user);

    $ie->getTextBox('name:', "password")->SetValue($pass);

    $ie->getButton('caption:', "Login")->Click;

    # Navigate to the add content page
    $ie->gotoURL('https://example.com/home/content.php');

    $ie->getButton('caption:', "Create new")->Click;

    ## Fill in Content Page
    $ie->getTextBox('name:', "name")->SetValue($title);
    $ie->getSelectList('name:', "company_id")->SelectItem("ACA");

    # IE fails on 6.00.2800.1106 whether or not a file is uploaded
    # Replace the
    # $ie->getTextBox('name:', "filename")->SetValue($upload);
    $ie->getTextArea('name:', "note")->SetValue("Sample comment on the upload");

    # Now we sleep to see at least 30 seconds to get the Post error, then click the "Save" button
    sleep($wait);
    $ie->getButton('caption:', "Save")->Click;

    # Summarize the output and quit IE so we always start from a known state
    $output=substr($ie->PageText(),0,40);
    print $output;

    $ie->closeIE();

The code starts up IE and walks it through the first few panes of the application until the point where the error has been known to occur. It’s evoked as, say: perl ieautomate.pl 5 where the last argument is the number of seconds to wait before the ultimate submit. When run with a short wait, like 5 seconds, the content is successfully posted. With a wait of 30 or 40 seconds, the submit fails.

Running this is really cool, like some poltergeist has taken over the machine. I can’t wait to use Win32::IEAutomation to check airline ticket prices, etc.

Diagnosing the SSL problems

This breaks down into two steps, a) getting VmWare host-only networking set up to route through the host so we can then b) run sslsniff on the traffic and look inside the packets.

A) Getting VmWare routing set up

Thanks to the folks at Cyberciti.biz for getting me on the right track. Their post on the matter is largely correct except that:

  • You need to run: echo 1 > /proc/sys/net/ipv4/ip_forward and add that config to /etc/sysctl.conf
* You would probably want to edit /etc/vmware/vmnet1/dhcp/dhcpd.conf to include: 

    option routers 192.128.2.1;
    option nameserver (_real ip of nameserver_)

It turns out this step is totally unnecessary. I’d intended to use the routing from the host’s eth0 interface to the guest’s vmnet1 subnet to run Mike Benham’s sslsniff. While SSLSniff works great in such a setup, if sufficently munges up the SSL traffic that it doesn’t aid in addressing the MSIE problem, in fact, it pretty well makes it go away.

B) Analyzing traffic with Ethereal/Wireshark

Ack, I’m getting tired so sorry this last part is so lame. What it comes down to is that three test cases were sufficient to reveal the crux of the problem.

  1. Firefox SSL POST—when using Firefox and taking about 30 seconds to fill out the form that gets POSTed, one can see ‘Encrypted Alerts’ coming down from the server about every ten seconds. The alerts are probably change_cipher_spec or more likely a close_notify. When the POST is sent, Firefox starts with an SSL ClientHello and sets up a whole new SSL session
  1. Automated MSIE post with no delays—when POSTing the form from the robot with no waits, everything works just great.
  1. Automated MSIE post with a 30s delay—while the robot is waiting to POST the form, the same Encrypted Alerts come down from the server, and the client responds with ACKs. But when the form is POSTed the client is trying to re-use the same SSL connection. The server simply replys with ACKS, and MSIE barfs

In case number 3 it’s pretty clear that the Magic Apache mod_SSL Incantation is not working, as close_notify messages are still getting sent from the server.

Over and out.

SSH SSL Shibboleth RSSscraper and other goodies from my old site

Posted by Peter Burkholder Thu, 14 Sep 2006 04:51:00 GMT

I’ve migrated a lot of my old sysadmin stuff from my old host. This includes the following stuff

Presentations

Notes

Some notes on work I do, which may or may not be useful to others

Articles

RSS Scraper Job Feeds

Since I’m looking for work with specific institutions of interest in the Washington, DC, area, I thought I could track jobs more easily if I could subscribe to the relevant RSS feeds. Alas, although any institution now has its “Job Opportunities” web page, they seldom have RSS feeds of those pages. So I decided to work out my Ruby and Regular Expression skills and build my own feeds. These are built from RubyForge RSSscraper, written by Monster or <a href=”http://www.dice.com”>Dice or their ilk, so I can’t compare this to what they offer. But I find it useful so far, although scraping HTML from the NSF has been really challenging. I’d recommend that one sign up with their email job notification service.

RSS Job Feeds