Closed
Description
This makes it appear as though filters are working even though they really aren't.
Here is the fix:
There is a check (AllTestCasesOfExecutableAreRun()) in CommandLineGenerator.cs if the testCases match all test cases. If they do you don't add any filters. I found this because my tests were taking a long time even when I was executing a few. It turned out all test cases were being run even though a filter was passed in as an argument to vstest.console.exe.
In TestExecutor.cs
public void RunTests(IEnumerable<string> executables, IRunContext runContext, IFrameworkHandle frameworkHandle)
{
try
{
InitTestEnvironment(runContext.RunSettings, frameworkHandle);
IList<Model.TestCase> allTestCasesInExecutables = GetAllTestCasesInExecutables(executables).ToList();
ISet<string> allTraitNames = GetAllTraitNames(allTestCasesInExecutables);
TestCaseFilter filter = new TestCaseFilter(runContext, allTraitNames, TestEnvironment);
List<TestCase> vsTestCases =
filter.Filter(allTestCasesInExecutables.Select(DataConversionExtensions.ToVsTestCase)).ToList();
>>>>>>>>>>>> allTestCasesInExecutables =
>>>>>>>>>>>>>>>>>> allTestCasesInExecutables.Where(tc => vsTestCases.Any(vtc => >>>>>>>>>>>>>>>>>> tc.FullyQualifiedName == vtc.FullyQualifiedName)).ToList();
>>>>>>>>>>>>>>>>>>> DoRunTests(allTestCasesInExecutables, allTestCasesInExecutables, runContext, frameworkHandle);
Change to:
>>>>>>>>>>>> var testCasesToRun =
>>>>>>>>>>>>>>>>>> allTestCasesInExecutables.Where(tc => vsTestCases.Any(vtc => >>>>>>>>>>>>>>>>>> tc.FullyQualifiedName == vtc.FullyQualifiedName)).ToList();
>>>>>>>>>>>>>>>>>>> DoRunTests(allTestCasesInExecutables, testCasesToRun, runContext, frameworkHandle);
}
catch (Exception e)
{
TestEnvironment.LogError("Exception while running tests: " + e);
}
}
Activity
bugfix: filtered test cases were run, even if their test results were…
csoltenborn commentedon Mar 15, 2016
Thanks for bug report and fix - that's a weird one... bugfix release will follow soon.