Extreme Mesh-Up using NUnit, PSExec, PowerShell and NetMon 3.2 to automate http traffic monitoring

Introduction

A few weeks ago I wrote a blog post about the Microsoft Visual Round Trip Analyzer (VRTA) tool and how we used it to tune the performance of our web application: Tuning Click-Once deployment performance with VRTA and PowerShell. In this follow-up article I am now demonstrating how I automated the analysis of the http traffic. The article also demonstrates one of the virtues of PowerShell: Meshing up different tools and technologies to automate a very specific tasks.

Figure 1: Dhaka traffic

Problem

I was looking for answers of the following questions:

• How can you automatically measure the http traffic caused by a remote client that is requesting certain pages from the web server?
• And what could be the most pragmatic solution for this problem?

I didn’t want to reinvent the wheel and needed to mesh something up quickly. The following set of screenshots demonstrates my solution. I was taking advantage of existing tools and PowerShell’s ability to “glue” it all together.

Toolbox

This mesh-up requires the following tools:

• NUnit - .NET unit testing framework to kick-off test sequence
• PSExec – Sysinternal’s remote command execution tool to start PowerShell on remote client computer
• PowerShell 2.0 CTP3 – Running the Internet Explorer load scenario, starting the Network Monitor capture and analyzing the capture file
• NetMon 3.2 – Microsoft Network Monitor tool for capturing http traffic frames
• VRTA – Optional tool to visualize the captured frames

Solution

Overview

The test sequence gets triggered by the execution of a NUnit unit test. I wrote a unit tests in C# that starts PowerShell on a remote machine using PSUnit from Sysinternals. I didn’t have time to figure out how to pass parameters to PowerShell via the PSExec command line, so I decided to launch my work horse PowerShell script as part of the PowerShell profile. This profile script executed during the startup of PowerShell. The PowerShell script starts nmcap.exe, which is the command line tool that the Microsoft Network Monitor 3.2 uses to automate the capture of network, in this case http, frames. Once the capture is started, PowerShell is then starting Internet Explorer and passing in the URL that IE is going to load initially. The network capture is running for about 60 seconds and then stops. Then the PowerShell closes IE and starts with the analysis of the *.cap file. It reads the number of frames that the capture contained. The script compares this number with a baseline value. If the value is not in the expected limits, then the script will delay the completion of the script and wait for 4 minutes. The script returns either right away or after the waiting period. On the other machine the call to launch PSExec returns. The unit test measures how long the roundtrip took. If it took longer than 5 minutes then the test fails, otherwise it passes. The following paragraphs provide some more details.

Nunit test to launch PowerShell on remote computer via PSExec

Figure 2: NUnit Test Runner executing unit test

The following source code defines a NUnit test that starts the process PSExec, which then starts PowerShell.exe on a remote computer.

   1: using System;

   2: using System.Collections.Generic;

   3: using System.Text;

   4: using System.Diagnostics;

   5:  

   6: using NUnit.Framework;

   7:  

   8: namespace NetMonUnitTest

   9: {

  10:     [TestFixture]

  11:     public class LauchRemotePSScript

  12:     {

  13:         [Test]

  14:         public void RemoteExecutionCompletesInLessThan5Minutes()

  15:         {

  16:             DateTime Start = DateTime.Now;

  17:  

  18:             Process PSExec = new Process();

  19:             ProcessStartInfo startInfo = new ProcessStartInfo(@"C:\BrowserTest\psexec.exe");

  20:             startInfo.Arguments = @"-i -u Administrator -p Pa$$w0rd \\Yellowtail powershell";

  21:             PSExec.StartInfo = startInfo;

  22:             

  23:             PSExec.Start();

  24:  

  25:             PSExec.WaitForExit(10 * 60 * 1000);

  26:  

  27:             DateTime End = DateTime.Now;

  28:             TimeSpan Duration = End - Start;

  29:             

  30:             Assert.Less(Duration.Minutes, 5, "Browser Performance test failed");

  31:  

  32:         }

  33:     }

  34: }

The unit test launches PSExec and waits for it to return. If it returns in less than 5 minutes than the Unit test passes, otherwise it fails.

Figure 3: PSExec started on server machine

PowerShell script snippet to start Netmon capture

The following snippet shows how to launch nmcap.exe to capture http traffic for about 60 seconds at the specified client IP address:

   1: "Starting netmon capture"

   2:  

   3: start-process -FilePath nmcap -ArgumentList "/network * /capture http AND ipv4.address == 192.168.1.71 /File c:\http.cap /stopwhen /timeafter 1min".Split()

Figure 4: PowerShell script in ISE

PowerShell script snippet to launch IE and navigate to a specific URL

The Netmon capture is running now. The script waits 10 seconds and launches Internet Explorer with a pre-defined URL.

Figure 5: Script launches Internet Explorer with pre-defined URL

   1: "Launching IE"

   2: start-process -FilePath "C:\Program Files\Internet Explorer\iexplore.exe" -ArgumentList "http://www.tellingmachine.com"

   3:  

   4: "Waiting for nmcap.exe to complete"

   5: wait-process -Name nmcap -Timeout 120

   6:  

   7: "Stopping IE"

   8: stop-Process -Name IExplore -Force

   9:  

  10: "Loading Capture File"

  11: $Buffer = Import-CaptureFile -Path "c:\http.cap"

  12:  

  13: "Parsing and calculating Frame count"

  14: $n = NumberOfFrames -Buffer $Buffer 

  15:  

  16:  

  17:  

  18:  

  19: $ExptectedFrameCount = 180

  20: $Tolerance = 20

  21: "Number of Frames: $n"

  22: "Expected        : $ExptectedFrameCount"

  23: "Deviation       : $Tolerance"

  24:  

  25:  

  26: if ($n -lt ($ExptectedFrameCount - $Tolerance) -or $n -gt ($ExptectedFrameCount + $Tolerance))

  27: {

  28:     "Test failed - Sleeping for 4 minutes"

  29:     start-sleep -Seconds 240

  30: }

  31: else

  32: {

  33:     "Test passed"

  34:     start-sleep -Seconds 10

  35: }

  36:  

  37: Stop-Process -Name PowerShell 

Figure 6: Netmon is capturing frames during the next 60 seconds

Figure 7: IE loaded successfully passed-in URL

PowerShell script snippet to analyze the Network Monitor capture file

Figure 8: Script analyzes capture file to get the number of http frames

The main PowerShell script calls a functions in a secondary script file called Import-NMCapture.ps1. One of the functions reads the binary data of the Netmon capture file and retrieves the number of frames.

   1:  

   2:  

   3:  

   4: Function Import-CaptureFile([string] $Path)

   5: {

   6:     $EOF = ""

   7:     $Bytes = Get-Content -Encoding Byte -Path $Path

   8:     return $Bytes

   9: }

  10:  

  11: Function ToInt( [Byte[]] $Buffer)

  12: {

  13:     [int] $Result = 0;

  14:     $Result += $Buffer[0]

  15:     $Result += $Buffer[1] * (0xFF + 1)

  16:     $Result += $Buffer[2] * (0xFFFF + 1)

  17:     $Result += $Buffer[3] * (0xFFFFFF + 1)

  18:     

  19:     return $Result

  20: }

  21:  

  22: Function ToString( [Byte[]] $Buffer)

  23: {

  24:     $Result = [String]::Empty;

  25:     $SB = New-Object -TypeName "System.Text.StringBuilder"

  26:     

  27:     foreach ($c in $Buffer)

  28:     {

  29:         $SB.Append([char][int]$c) | Out-Null

  30:     }

  31:     $Result = $SB.ToString()

  32:     return $Result

  33: }

  34:  

  35: Function NetMonCapFileMagicNumber([Byte[]] $Buffer)

  36: {

  37:     ToString $Buffer[0..3] 

  38: }

  39:  

  40: Function FrameTableOffset([Byte[]] $Buffer)

  41: {

  42:     ToInt $Buffer[24..27]

  43: }

  44:  

  45: Function FrameTableLength([Byte[]] $Buffer)

  46: {

  47:     ToInt $Buffer[28..31]

  48: }

  49:  

  50: Function NumberOfFrames([Byte[]] $Buffer)

  51: {

  52:     [int] $size = FrameTableLength -Buffer $Buffer

  53:     return $size / 4 

  54: }

  55:  

  56: Function FrameTable([Byte[]] $Buffer, [int] $Offset, [int] $Length)

  57: {

  58:     

  59:     $Result = $Buffer[ $Offset .. ($Offset + $Length)]

  60:     $Result 

  61: }

  62:  

  63: Function FrameOffsets([Byte[]] $Buffer, [int] $Offset, [int] $Length)

  64: {

  65:     $FrameTable = FrameTable -Buffer $Buffer -Offset $Offset -Length $Length

  66:     $Result = New-Object -TypeName "System.Collections.ArrayList"

  67:     [int] $FrameOffset = 0

  68:     

  69:     for( $i=0; $i -lt $FrameTable.Length - 4; $i += 4)

  70:     { 

  71:         $FrameOffset = ToInt $FrameTable[$i..($i + 4)]

  72:         $Result.Add($FrameOffset) | Out-Null

  73:     }

  74:     return $Result.ToArray()

  75: }

Figure 9: NUnit test failed. The actual number of frames didn’t meet the requirement

Figure 10: NUnit test passed. The number of frames met the requirements

Figure 11: Capture file in NetMon. Analyzing frames.

Figure 12: Capture file in VRTA. Graphical analysis of frame content.

Download

The source files of the scripts and NUnit test project can be downloaded here: AutomateNetMon.zip

Ausblick

I was quite surprised how relatively easy it was to put this together. It has quite some potential not just for capturing http frames, but also for load testing web servers. Here is a collection of ideas that can be added to round up my prototype:

• Use PowerShell 2.0 remoting instead of PSExec
• Parameterize the URLs and the wait periods between the navigations
• Take advantage of a web testing framework like WebAii to access AJAX requests
• Use WebAii to take screen shots of the rendered pages in the web browser
• Support IE, Firefox and Safari
• Figure out how to determine from outside the browser when a page is completely loaded and start measure the page load times in the browser
• Capture the files that a page request returns as actual files
• Resetting the browser cache from PowerShell

I strongly believe that PowerShell is a series thread to web server load testing frameworks and I would like to spend some more time working on these ideas. Stay tuned!