Currently, the ForEach activity in ADF (or Synapse) does not break out when an inner activity fails. There are several ways you can force the ForEach to break out, the most common is to cancel the pipeline run when an inner activity fails and there are already many blogs out there that cover this. My requirement is slightly different and that is what I will show here.
Consider a scenario where you have a ForEach activity which has multiple inner activities per iteration, and you don’t want to cancel the pipeline run on failure, but you want to continue the pipeline run and run a subset of the activities instead. The following is a very generic example which doesn’t do anything useful but is purely to demonstrate the required behaviour.
In my pipeline I have a LookUp activity and then a ForEach actvity which iterates over the output of the LookUp. I have also added a variable to keep track of the status of the ForEach activity. This is all shown below.
Inside the ForEach I have two activities, an IfCondition and a Wait (this is for demo purposes and could be anything else). The IfCondition checks the value of the variable “ForEachStatus” using the expression @not(equals(variables(‘ForEachStatus’), ‘Failed’)) and only executes the inner activities if this expression equates to true (so when the variable is not set to “Failed”). Note the output from the IfCondition to the Wait is “Completion”, this ensures the Wait activity in the ForEach is executed even when the IfCondition fails so the pipeline can continue to run.
Inside the IfCondition is the inner activity (Wait activity used for demo purposes only) and a SetVariable activity which is executed only when the inner activity fails, the outcome of these two activities determines how the remainder of the pipeline is executed.
If the inner activity is successful, the ForEach moves onto the next activity outside the IfConditon and then to the next item in the ForEach. If the inner activity fails the “ForEachStatus” variable is set to “Failed” and the pipeline continues to the next activity outside the IfCondition, but when it gets to the next item in the ForEach the IfCondition equates to false so the activity inside the IfCondition is no longer executed for the remainder of the pipeline run.
You can see this in the two outputs below, the first output on the left is without any failures and the second output on the right is when the inner activity has failed. In the first output you can see the inner activity is executed multiple times but in the second output it shows that as soon as the inner activity fails the SetVariable activity is executed and this stops the inner activity from being executed again, but the outer activity still runs and the pipeline is still successful.
One thing to note is that this set up works best when the ForEach is set up as sequential, the behaviour is otherwise unpredictable when running in parallel. And this is just one way of handling failures in the ForEach activity. For example, you could execute a completely different set of activities in the ForEach once you get a failure, do this by adding activities to the False part in the IfCondition. Or you could have additional activites after the ForEach that continue to run if an inner activity fails, to do this you would need to set the output from the ForEach activity as “Completion”. Finally, just a reminder that you can cancel the pipeline run as soon as an inner activity fails in a ForEach, and there are several blogs out there that show this.
