Yaml: How to exclude some tests in VSTest@2

Question

Currently, we're moving to AzureDevOps. We have some unit tests that fails on the agent but we don't have the capacity to fix them immediately. Until we fix them we'd like to exclude them from the pipeline and set a policy that all the test should pass, then fix those one-by-one.

Let me say, I have 4 exes, namely ABE_uTest.exe, BBE_uTest.exe, CBE_uTest.exe and DBE_uTest.exe. I'd like to exclude BBE and CBE. On the documentation page (AzureDevOps VsTest) I've found how to exclude files.

testAssemblyVer2: | # Required when testSelector == TestAssemblies
# *test.dll
# !*TestAdapter.dll
# !\obj*

So I assume simply a *_uTest.exe !BBE* !CBE* would work. But I miss something, because I can't make it work. Either it filters nothing, or everything. Here's the corresponding part of the yaml.

- task: VSTest@2
  displayName: VsTest - testAssemblies
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: '*_uTest.exe
      !*BBE*'
    searchFolder: '$(BuildPlatform)_$(BuildConfiguration)\testbin\'

The testAssemblyVer2 would be the setting. I've tried with only one exclude binary to make it more simple. I tried with and without ' characters at the beginning and the end. Also tried in one line and in separate lines but with no success. I always got this message:

##[warning]No test sources found matching the given filter '*_uTest.exe !BBE_uTest.exe'

What is the correct syntax to exclude files in AzureDevOps using yaml?

soundflix · Accepted Answer · 2024-01-15 23:41:16Z

6

The notation is quite specific, you are missing a | and need to place it on several lines (other notations might exist, this works). Added a **\ for searching in every possible folder:

- task: VSTest@2
inputs:
  testSelector: 'testAssemblies'
  # note the |
  testAssemblyVer2: |
    **\*_uTest.exe
    !**\*BBE*

Why the |? It signals that you want to input several different items. 'Something something2' means look for a very long name with a space in it. Without the | several lines are seen as 1 input. See the documentation.

edited Jan 15, 2024 at 23:41

soundflix

3,13312 gold badges19 silver badges39 bronze badges

answered Nov 18, 2021 at 16:28

Taco Verhagen

2521 silver badge6 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

osim_ans · Accepted Answer · 2024-02-12 07:27:58Z

4

I was able to skip the tests using the testFiltercriteria input property of VsTest task, as shown below:

- task: VSTest@2
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\*_AlluTests.dll
    testFiltercriteria: 'FullyQualifiedName!~TestCase1&FullyQualifiedName!~TestCase2'

answered Feb 12, 2024 at 7:27

osim_ans

4873 silver badges14 bronze badges

Collectives™ on Stack Overflow

Yaml: How to exclude some tests in VSTest@2

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related