Exception discovering xUnit.net tests with Visual Studio Online build server

Question

I am using Xunit.net with Visual Studio Online hosted build. My tests are being discovered and run fine both locally and on the build server. But on the build server, I get this exception (causing the build to reach a "Partially Succeeded" state). This is odd since all my tests are in fact being discovered and run.

[xUnit.net 00:00:01.3170293] Exception discovering tests from C:\a\bin\xunit.runner.visualstudio.testadapter.dll: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

File name: 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

File name: 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'

WRN: Assembly binding logging is turned OFF.

To enable assembly bind failure logging, set the registry value HKLM\Software\Microsoft\Fusion!EnableLog to 1.

Note: There is some performance penalty associated with assembly bind failure logging.

To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

   at System.Reflection.RuntimeAssembly.GetExportedTypes(RuntimeAssembly assembly, ObjectHandleOnStack retTypes)

   at System.Reflection.RuntimeAssembly.GetExportedTypes()

   at Xunit.Sdk.Executor.EnumerateTests..ctor(Executor executor, Object _handler)$$RethrowMarker$$   at ExceptionExtensions.RethrowWithNoStackTraceLoss(Exception ex)

   at Xunit.RemoteAppDomainManager.CreateObjectTObject

   at Xunit.Xunit1Executor.EnumerateTests(ICallbackEventHandler handler)

   at Xunit.Xunit1.Find(Predicate`1 filter, Boolean includeSourceInformation, IMessageSink messageSink)

   at Xunit.Xunit1.Find(Boolean includeSourceInformation, IMessageSink messageSink)

   at Xunit.XunitFrontController.Find(Boolean includeSourceInformation, IMessageSink messageSink)

   at Xunit.Runner.VisualStudio.TestAdapter.VsTestRunner.GetTests(IEnumerable`1 sources, IMessageLogger logger, XunitVisualStudioSettings settings, Stopwatch stopwatch)

See http://go.microsoft.com/fwlink/?LinkId=254169

I have the following nuget packages installed in the test project:

• xunit 1.9.2
• xunit.runner.visualstudio 0.99.2

Other unit testing frameworks, such as MS Test and NUnit, are working without problems. This makes me think the issue is with Xunit.net rather than with Visual Studio Online.

I have also opened an issue at the xUnit.net GitHub, but it remains unsolved.https://github.com/xunit/xunit/issues/47

How can I get this working? Does anyone know of a workaround? Could I somehow suppress the error message?

Answer 1

The testrunner tries to discover the unit tests in xunit.runner.visualstudio.testadapter.dll. Why? Because it matches the default test sources spec of *.test*.dll.

When changing the default test source spec to *.tests.dll or something else more specific, it will work.

Source: http://erictummers.wordpress.com/2014/02/11/execute-xunit-tests-on-hosted-build-controller/