Automation: issue when filtering with smart values during variable creation

[Thibault DUCHATEAU]

Hi there!
I am having a lot of trouble getting the following filter to work.

Context: I am making an API call that returns a list of messages.

I want to filter only the items on the following fields:
- metadata.event_type=“hardCodedString”
- metadata.event_payload.field=“{{value}}” <= value is (supposed to be) a string

A few notes:
- some items do not have a "metadata" field.
- some items do not have the "event_payload.field" field.

Here is the expression I am currently using:

{{#webResponse.body.messages}} 
  {{#if(equals(metadata.event_type, "hardCodedString"))}}
    {{#if(equals(metadata.event_payload.field, value))}}{{ts}}| {{/}} 
  {{/}} 
{{/}}

Since the field “value” is unique (from a business perspective), I should only get one result or none at all.

However, I regularly end up with inconsistent results: items without metadata, items where the event_type does not match, and items where the field value (within event_payload) does not match.

Does anyone have any idea where the problem might be coming from?

Thanks!

[Bill Sheboy]

Hi @Thibault DUCHATEAU -- Welcome to the Atlassian Community!

There is a known limitation of long-format list iteration where, once inside the iterator, only data from that scope and lower is accessible. 

Thus, your value cannot be "seen" inside to perform the filtering.  The Atlassian automation team knows about this, and according to other posts, has tried several times to solve this without success.  And, there are no built-in, JSON filtering functions for webResponses yet in rules.

Possible workarounds depend upon what you are trying to do with the filtered results:

• Want to process them one by one
  • Use an advanced branch to iterate over the records and use Smart Value Conditions to test the values
• Want to process them as a batch
  • There is a complicated workaround where the list contents are flattened out into a delimited text string, split back into a list for inline list iteration, filtered using a dynamically built regular expression with the match() function, and then the results extracted using text functions
  • If you want to try this approach, please see this article I wrote on the topic: https://community.atlassian.com/forums/Automation-articles/Automation-concepts-Dynamic-searches-within-a-list/ba-p/2834235

Kind regards,
Bill