Wednesday, 1 August 2012

Parsing MSTEST results (.trx files) programmatically (in C#)

 

Hello, it’s been quite awhile since I’ve posted anything; I’ve been quite busy.  I just felt like sharing this.  This is an extension of other people’s ideas, but I’ve taken it slightly further and thought it might be useful to somebody.

I came across a situation where I needed to programmatically examine a .trx file for specific strings.  The .trx file was large (31MB) because it contained the output of around 5000 unit tests.  Around 800 of those tests were failed tests, and they were failing for a specific reason (which is not important now).  The point is, I wanted to programmatically examine the error messages that were being output, and do something (which is also not important now), based on certain strings that might be found in the error message of each failed test.

Searching led me to this post in Rasheed’s blog, which was a great starting point for what I wanted to do.  As described in that post, first I had to:

Use XSD.EXE (see MSDN here) to generate classes based on the schema of VSTST.XSD. 

(XSD.EXE can be found in the Windows SDK).

(If you’re running Visual Studio 2010, then VSTST.XSD can be found in the Xml\Schemas directory on your local drive; in my case that was):

C:\Program Files (x86)\Microsoft Visual Studio 10.0\Xml\Schemas

So I copied VSTST.XSD to a working directory on my C:\ drive, let’s call it C:\Projects\VSTST-CLASSES.

Then I executed this command:

C:\Projects\VSTST-CLASSES>"C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools\xsd.exe" vstst.xsd /c

That generated a file called VSTST.CS.  In Visual Studio, in a new console application, I chose “Add Existing Item” and added that .CS file to the project to give me access to the generated classes.

Then (as described in the user comments in Rasheed’s blog ), I had to go into VSTST.CS, which now contains a load of classes that can be used to deserialize your test data, and comment out the “Items” properties of the GenericTestType and CodedWebTestElementType, to get the test results data to deserialize without error.

The end result of all this can be seen in the following C# code (which is a variation of Rasheed’s, but, I don’t care about outputting it to Excel; I’m just trying to find certain strings in the error message, probably so I can output that text to a separate output file, which will probably be a text file in any case).  Modify as desired to find the string you want in the error message of your failed test.  This code deserializes your test results and lets you loop through them and do something  based on the contents of the error message on failed unit tests.

using System.IO;

using System.Xml;

using System.Xml.Serialization;

 

namespace Utility

{

 

    class Program

    {

        static void Main(string[] args)

        {

            var fileInfo = new FileInfo(@"C:\thePathToMyGiganticTrxFile\DBarrows@R9D0YGF 2012-07-16 20_08_01.trx");

 

            var fileStreamReader = new StreamReader(fileInfo.FullName);

 

            var xmlSer = new XmlSerializer(typeof(TestRunType));

            var testRunType = (TestRunType)xmlSer.Deserialize(fileStreamReader);

 

            foreach (var itob1 in testRunType.Items)

            {

                var resultsType = itob1 as ResultsType;

                if (resultsType == null) continue;

 

                foreach (var itob2 in resultsType.Items)

                {

                    var unitTestResultType = itob2 as UnitTestResultType;

 

                    if (unitTestResultType == null) continue;

 

                    // these are not used, but they could be to identify the output:

 

                    // var id = unitTestResultType.testId;

                    // var testName = unitTestResultType.testName;

 

                    var outcome = unitTestResultType.outcome;

 

                    if (outcome != "Failed") continue;

 

                    var items = unitTestResultType.Items;

 

                    if (items == null || items.Length <= 0) continue;

 

                    // now we know we have a failed test; look for the desired string(s) in the error message

                    var outputType = (OutputType) unitTestResultType.Items[0];

                    var errorInfo = outputType.ErrorInfo;

                    var message = errorInfo.Message;

                    var text = ((XmlNode[]) message)[0].InnerText;

 

                    if (text.ToLower().Contains("some_pertinent_string"))

                    {

                        // do something special if the error message contains some particular string,

                        // e.g. parse the error message further and output as desired

                    }

                }

            }

        }

    }

}

 

 

 

8 comments:

  1. Is the VSTST.CS file generated by xsd.exe tool unique for every .trx files? I have to prepare a tool which parses .trx file at some dynamically provided location each time it runs. Any pointers regarding the same??

    ReplyDelete
  2. Hi Sourodeep, it's been awhile since I did that work. But if I understand your question, looking at the above C# code I posted, it shouldn't be difficult to pass in the path to the file(s) as a parameter using the System.IO namespace, e.g. http://msdn.microsoft.com/en-us/library/system.io.directory.getfiles.aspx . Good luck, -Dave

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. Is there any way or tool which compares two trx files i.e last test result and new test result and identifies the tests which were passing in last build and are failing in new build.

    ReplyDelete
    Replies
    1. @Akanksha, sounds like you'd have to code that by hand. The System.IO namespace in C# should allow you to open both files and parse them in creative ways to get the result you want. https://msdn.microsoft.com/en-us/library/system.io.file.readalllines%28v=vs.110%29.aspx, and/or use various XPath or XSLT approaches. That's all I can really suggest for a question like that. On the other hand, I wonder, why would you need such a tool? If a test is failing, it should be fixed, and it doesn't really matter that it was passing in the previous build...

      Delete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Hello !!!

    Is it possible to access the results (which are going to be written in .trx file) during the test run before the final .trx file gets generated . If so then how this can be acheived?

    Thanks and Regards
    Srinivas

    ReplyDelete
    Replies
    1. Presently the results are retrieved for the trx file which is already generated. My question is before the generation of the .trx file , is it possible to access the results which is going to be written in to the .trx file? If so then how this can be done programatically?

      Thanks and Regards
      Srinivas

      Delete