Construct a condition in Azure DevOps using 'each'

Question

In Azure DevOps (YAML pipeline), we have a stages that should be run only after another set of stages have been skipped.

In the example below, the parameter copyStages_UAT can be amended by users when triggering a manual run, meaning it's impossible to hard-code the dependsOn and condition properties, so necessitating the use of the directive each.

- template: ../Stages/stage--code--depoly-to-environment.yml
  parameters:
    name: Deploy_PRD_UKS
    displayName: Deploy PRD - UK South
    dependsOn:
    - ${{ each uatStage in parameters.copyStages_UAT }}:
      - Roll_Back_${{ uatStage.name }}
    variables:
    - template: ../Variables/variables--code--global.yml
    - template: ../Variables/variables--code--prd.yml
    environment: PRD

This stage above works in a pipeline, however because a successful run results in stages defined in dependsOn being skipped, sadly then Azure DevOps will also skip this stage.

To counter this, I'm trying to add a condition to check whether or not the previous stages were all skipped.

condition: >-
  and(replace(
    ${{ each uatStage in parameters.copyStages_UAT }}:
      eq(dependencies.Roll_Back_${{ uatStage.name }}.result, 'Skipped'), 
  ), ', )', ' )')

Unfortunately though, it seems as though I cannot use the directive each in this context -

The directive 'each' is not allowed in this context. Directives are not supported for expressions that are embedded within a string. Directives are only supported when the entire value is an expression.

As condition can only be a string, how can I leverage expressions and/or directives to construct my desired condition?

Example of desired YAML

Assuming the following value was given for the parameter copyStages_UAT -

- name: UAT_UKS
  displayName: UAT - UK South
- name: UAT_UKW
  displayName: UAT - UK West

This is how the YAML should be compiled. I'm not worried out the format of the condition, as long as the relevant checks are included.

- template: ../Stages/stage--code--depoly-to-environment.yml
  parameters:
    name: Deploy_PRD_UKS
    displayName: Deploy PRD - UK South
    dependsOn:
    - Roll_Back_UAT_UKS
    - Roll_Back_UAT_UKW
    condition: >-
      and(
        eq(dependencies.Roll_Back_UAT_UKS.result, 'Skipped'),
        eq(dependencies.Roll_Back_UAT_UKW.result, 'Skipped')
      )
    variables:
    - template: ../Variables/variables--code--global.yml
    - template: ../Variables/variables--code--prd.yml
    environment: PRD

concision · Accepted Answer · 2022-04-28 23:54:48Z

Azure DevOps Pipelines does not have a particularly good way for solving this. However, and(...), join(delimiter, ...) and 'Filtered Arrays' can be used to hackily accomplish this.

Observe that the following condition could be rearranged:

and(
  eq(dependencies.Roll_Back_UAT_UKW.result, 'Skipped'),
  eq(dependencies.Roll_Back_UAT_UKX.result, 'Skipped'),
  eq(dependencies.Roll_Back_UAT_UKY.result, 'Skipped'),
  eq(dependencies.Roll_Back_UAT_UKZ.result, 'Skipped')
)

and(
  eq(dependencies.Roll_Back_
  UAT_UKW
  .result, 'Skipped'), eq(dependencies.Roll_Back_
  UAT_UKX
  .result, 'Skipped'), eq(dependencies.Roll_Back_
  UAT_UKY
  .result, 'Skipped'), eq(dependencies.Roll_Back_
  UAT_UKZ
  .result, 'Skipped')
)

Or more abstractly, where PREFIX=eq(dependencies.Roll_Back_ and SUFFIX=.result, 'Skipped'):

and(
  <PREFIX>
  UAT_UKW
  <SUFFIX>, <PREFIX>
  UAT_UKX
  <SUFFIX>, <PREFIX>
  UAT_UKY
  <SUFFIX>, <PREFIX>
  UAT_UKZ
  <SUFFIX>
)

With filtered arrays (NAMES=parameters.parameterName.*.name) to extract the name, an aggregation could then be written:

and(<PREFIX>${{ join('<PREFIX>, <SUFFIX>', <NAMES>) }}<SUFFIX>)

Thus:

condition: |
  and(
    ne(dependencies.Roll_Back_${{ join('.result, ''Skipped''), ne(dependencies.Roll_Back_', parameters.copyStages_UAT.*.name) }}.result, 'Skipped')
  )

But there are some obvious caveats with this:

If there is 0 elements in parameters.copyStages_UAT, then the expression would evaluate to and(dependencies.Roll_Back_.result, 'Skipped') which could be non-sensical.
and(...) requires a minimum of 2 arguments, so this could potentially fail if there is only 0 or 1 expression(s). To circumvent this, True can be supplied as the first and second argument such that the expression is always valid. If your logic instead requires or(...), then use False instead of True to keep the meaning consistent.

Therefore, you may need to protect against these scenarios occurring with a modified check:

${{ if eq(length(parameters.copyStages_UAT), 0) }}:
  condition: false
${{ else }}:
  condition: |
    and(
      True,
      ne(dependencies.Roll_Back_${{ join('.result, ''Skipped''), ne(dependencies.Roll_Back_', parameters.copyStages_UAT.*.name) }}.result, 'Skipped')
    )

Mengdi Liang · Accepted Answer · 2020-09-22 07:56:57Z

0

Updated:

To summary your demand, you are looking for a expression that you can use it in condition while the dependsOn value are dynamic. And this stage should run only after another set of dependent stages are all skipped.

As far as I know and tested, this can not be achieved via each.

For further confirmation, I discussed this scenario with our pipeline PM who is more familiar with each and YAML pipeline.

Same as me, he also think this isn't possible to achieve. Because ${{ each }} expects to be the outermost part of a mapping key or value, so you can’t nest it into a condition string.

Work around:

You could fake it by having a hard-coded stage which depends on the dynamic list, figures out if they were all skipped, and sets an output variable. Then the real final job would only depend on that “decider” job, and its condition would depend on the contents of the output variable.

Figuring out that all upstream dependencies were skipped is something of an exercise for you. You might be able to dynamically construct one step per stage. Those steps map the stage’s status to a variable with a known name scheme. Then the final (hardcoded) step iterates the environment variables of the known name scheme and decides whether the next stage should proceed.

And... yes, I am aware how ugly that sounds ☹

edited Sep 22, 2020 at 7:56

answered Sep 16, 2020 at 9:42

Mengdi Liang

19.3k2 gold badges35 silver badges43 bronze badges

6 Comments

David Gard Over a year ago

Thanks for your reply. In this case, I only want to run one stage if the stages it depends upon have been skipped. So using your examples above, both images are correct. However, I cannot use always() because that would mean that the DeployEurope stage would always be run, even if avaiableazure and merlinazure had been run, and as mentioned it should only be run if those stages it depends on are skipped. I have updated my question to (hopefully) make it clearer.

Mengdi Liang Over a year ago

@DavidGard, appreciate for this detailed explanation. Got that. That is clear for me now. Would share you suggestions once I have update here.

Mengdi Liang Over a year ago

@DavidGard, As far as I know, this is not possible to use each directly in condition. To avoid misleading you, I discussed this with our program manager who are most familiar with the function of each for further confirmation. Same result, we are all think as of now, you need achieve that by using work around. See my update answer.

David Gard Over a year ago

Thanks for the confirmation. That is not ideal, and I'm not sure that the workaround is going to be feasible. As far as I am aware, you can only share output variables either between tasks in a job, or between jobs in a stage (using isOutput=true) - indeed the documentation specifically says "Multi-job output variables only work for jobs in the same stage."

Mengdi Liang Over a year ago

@DavidGard Ah, we have released a new feature in sprint 168, so that users can access output variable across stages. See this feature timeline doc: learn.microsoft.com/en-us/azure/devops/release-notes/2020/…

|

Estanislao Bosch · Accepted Answer · 2024-07-08 10:53:15Z

0

Response from @concision is what works for me. The only change I suggest to their description is changing

and(<PREFIX>${{ join('<PREFIX>, <SUFFIX>', <NAMES>) }}<SUFFIX>)

by

and(<PREFIX>${{ join('<SUFFIX>, <PREFIX>', <NAMES>) }}<SUFFIX>)

I couldn't make a comment to their answer as I need to have 50 reputation. That is why I'm posting this as a new answer

answered Jul 8, 2024 at 10:53

Estanislao Bosch

1

Comments

A. K. · Accepted Answer · 2025-07-24 10:12:39Z

0

You can construct a condition like this to declare stages only known at runtime:

Define parameters

parameters:
- name: TargetChoice
  displayName: '[TargetChoice] Select target environment.'
  type: object
  default: DEV1,DEV2,DEV3
.
.
.

Define stages

- ${{ each currentEnvironment in split(parameters.TargetChoice,',') }}:
  - stage: Code_Deploy_To_${{ currentEnvironment }}
.
.
.

Define stage dependent on all each stage

.
.
.
- stage: Final_Email
  condition: eq('${{ parameters.IsScheduled }}', 'true')
  dependsOn:
  - ${{ each currentEnvironment in split(parameters.TargetChoice,',') }}:
    - Code_Deploy_To_${{ currentEnvironment }}
  jobs:
.
.
.

answered Jul 24 at 10:12

A. K.

415 bronze badges

Collectives™ on Stack Overflow

Construct a condition in Azure DevOps using 'each'

Example of desired YAML

4 Answers 4

Comments

6 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Example of desired YAML

4 Answers 4

Comments

6 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related