A few days ago I blogged about a PowerShell script that imports a bulk of NUnit test result XML files into a TeamCity Continuous Integration Server. This blog post takes this concept a bit further by using TeamCity to start a “fake” build synchronously with a NUnit test run that was started on an external test system. The “fake” TeamCity build will then wait for a file system event and trigger the XML file import after the tests are finished. This way it seems like the TeamCity test build is running in real-time and executing the tests. The TeamCity build task that is waiting for a file system event is using a NAnt script that calls a PowerShell script. The PowerShell script takes care of the file system event handling.
Figure 1: The early stages of Windows 8
High level overview
Calling StartTeamCityBuildWithCurl.ps1 will kick off the TeamCity build. In my case I am using a batch file that you can find in the download zip package. The batch file passes build parameters to the TeamCity build. The build then uses the ImportTestResults.build NAnt script to call the BuildTestResultsWatcher.ps1 PowerShell script. This script sets up a file system event handler and calls ProcessXML.ps1 to import a the fresh test result XML file.
Script file listings
The main script files that are involved in this workflow chain are listed here:
Set-StrictMode -Version "Latest"
$DebugPreference = "Continue"
$URL = "`"http://localhost:8887/httpAuth/action.html?add2Queue=bt3&env.name=ExternalBuildID&env.value=$BuildNumber`""
Write-Debug "Starting teamcity via cURL and teamcity REST API"
$result = start-process -filePath "C:\TestResultsImporter\Tools\curl\curl" `
-ArgumentList "--user", "autoadmin:LuckyYou", $URL `
-RedirectStandardError "error.txt" `
-RedirectStandardOutput "output.txt" `
<?xml version="1.0" ?>
<project name="TestResultsImporter" default="build" xmlns="http://nant.sf.net/release/0.90/nant.xsd">
<target name="build" depends="send.nunit.report.import.message.to.teamcity" />
<target name="wait.for.testresults.xml.file" description="Starts blocking PowerShell script that waits for FileSystemEvent and processes XML file">
<echo message="Starting PowerShell Script to wait for testresult.xml file and then convert the format"></echo>
<exec program="PowerShell" commandline="-file "C:\TestResultsImporter\Scripts\LatestResultsImporter\TestResultsFileWatcher.ps1"" />
<target name="send.nunit.report.import.message.to.teamcity" depends="wait.for.testresults.xml.file" description="Sends message via standard output to teamcity to trigger the import of a NUnit test result report">
<echo message="##teamcity[importData type='nunit' path='C:\TestResultsImporter\Pickup\TestResults.xml' parseOutOfDate='true']"></echo>
$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = "C:\TestResultsImporter\PublishedTestResults\v0.1"
$watcher.IncludeSubdirectories = $true
$watcher.EnableRaisingEvents = $true
$watcher.Filter = "TestResults.xml"
function Process-TestResultsFile([string] $BuildFolder)
$TestResultsFilePath = Join-Path -Path $watcher.Path -ChildPath $BuildFolder
Write-Host "Found TestResults.xml"
. "C:\TestResultsImporter\Scripts\Common\ProcessXML.ps1" `
-BeforeXMLFilePath $TestResultsFilePath `
-AfterXMLFilePath "C:\TestResultsImporter\Pickup\TestResults.xml" `
Write-Host "Error converting XML"
$ProcessTestResultFileJob = Register-ObjectEvent -InputObject $watcher -EventName "Created" -SourceIdentifier "TestResultsFileCreatedEventIdentifier"
$CreatedEvent = Wait-Event -SourceIdentifier "TestResultsFileCreatedEventIdentifier"
$BuildFolder = $CreatedEvent.SourceEventArgs.Name
Remove-Event -SourceIdentifier "TestResultsFileCreatedEventIdentifier"
Process-TestResultsFile -BuildFolder $BuildFolder
Unregister-Event -SourceIdentifier "TestResultsFileCreatedEventIdentifier"
param([string] $BeforeXMLFilePath, [string] $AfterXMLFilePath, [string] $XMLTemplatePath)
function Load-XML([string] $Path)
$xml = [xml](get-content -Path $Path -Force)
$FUTestResultXML = Load-XML -Path $BeforeXMLFilePath
#Do some processing, if required.
Step By Step setup guide
The first important step is to extract the zip file that is available as download and copy its content into the C:\ root folder of your PC.
The ToImport folder contains the NUnit test results XML files for each particular build.
Figure 2: Test results file to import
The PublishedTestResults folder is being watched by the FileSystemWatcher script, which is in this case triggered by the TeamCity NAnt build task. The associated TeamCity build will not complete until you copy manually one of the test result folders into the PublishedTestResults directory.
Figure 3: Folder being watched by the FileSystemWatcher script
You need to create a very simple build configuration. Add a NAnt build runner, select the NAnt executable in the tools folder and hook it up to the ImportTestResults.build file.
Figure 4: Create a new TeamCity project and a new build configuration
Figure 5: ImportTestResults.build is now hooked up
Running the import
First start the ReRun.bat file. The file can be found in the folder C:\TestResultsImporter\Scripts\LatestResultsImporterMake. Make sure that the build number you pass in is valid. This batch file will trigger the TeamCity build via REST API and the cURL tool.
Figure 6: The rerun.bat file kicked off a new TeamCity build
The TeamCity NAnt build task is calling a PowerShell script that then waits for the test result XML file to be copied into the PublishedTestResults directory.
Figure 7: Waiting for NUnit tests to be complete
To simulate the completion of a test run we can just manually copy folder 77 for example into the PublishedTestResults directory. The PowerShell script that is waiting for a new XML file is going to pick it up and then import the XML file via TeamCity service messages into TeamCity.
Figure 8: New XML file is picked up and imported
Watch the following YouTube video to see the import magic in action.
The scripts and sample data can be downloaded here: C_TestResultsImporter.zip
The downside of this approach is that it consumes a TeamCity agent for the duration of the NUnit test run. During the wait, the Agent is actually idle. This approach is not scalable in the sense that results of tests that are running concurrently in different environments cannot be imported without the need of several agents.