MuleSoft’s ‘For Each’ Connectors Explained
Estimated reading time: 5 minutes
MuleSoft® provides many connectors to help businesses develop API-led connectivity solutions. Out of all of those connectors, the “For Each” loop is a commonly used connector. This connector iterates through each element of a collection, or in other words, each element of an array.
There are two different For Each connectors that MuleSoft provides. There is the “For Each” and the “Parallel For Each.” They have key different functionalities and serve different purposes. Knowing when to use each one is crucial for the success of the API’s you are building.
When to use ‘For Each’
The For Each connector gets an array of elements and processes, each one at a time, so if there are 60 elements and each one takes one-second to process, making the total run for this connector one-minute. That’s something to keep in mind when using this connector since it could slow down your entire flow.
This is why the For Each connector should be used with a small amount of elements, or when performance isn’t critical.
The other thing to keep in mind when using this For Each connector is that if one element fails to process, the connector stops processing the rest of the records and moves on to the next connector. For example, let’s say that out of 60 records the connector fails at record 29, the remaining 31 records won’t process since the connector failed on record 29.
Another important piece to keep in mind is that this connector doesn’t return a processed payload. Whichever payload is given to this connector gets passed through to the next connector.
In the above example, the Collection field in For Each is set to iterate over an array stored in payload.topics:.
Key ‘For Each’ Takeaways:
- The For Each connector should be used when there are a few items to loop for per call.
- Use when performance isn’t crucial.
- Use when one payload fails the rest should stop processing.
- Use when a processed payload isn’t returned.
When to use ‘Parallel For Each’
Since the For Each connector doesn’t return a processed payload, one of the common patterns with the connector is to use Object Store to store the processed data for each element. This can be used successfully, but is not recommended since the Object Store has limitations on how many records it can process on a timeframe. It also can fail if you try to get elements from it at a pace faster than its limits.
One of the ways to avoid this pattern is to use the Parallel For Each connector. With this connector each element is processed individually and the end result is all the payloads for each processed element are merged into one payload. You can then iterate through each one and run the logic that’s needed to run on each element. This case can’t be used for all scenarios, but it’s a way to avoid the Object Store pattern.
Parallel For Each works by processing each element in its own thread. There is a limit on how many threads it can run at one time (see the links below for more info). You can also set the limit yourself, which gives you flexibility and control on the amount of threads it can run at one time.
Parallel For Each processes all records and the result is their payload – regardless of failure or success – will return the same amount of payloads that match the elements given. This is useful when you want all records to process, regardless of the status. You can then use your own logic afterwards on what to do with success and failure payloads.
Another key component to the Parallel For Each connector is that the end results are returned as a combined payload. So it actually returns the payload and you can do with it what you want. The For Each connector doesn’t return a payload and you have to use a variable or Object Store in order to store and retrieve the payload processed on the connector.
Key ‘Parallel For One’ Takeaways:
- Avoid using Object Store in a loop, since Object Store drops messages sometimes.
- Use when performance is necessary.
- Use when all should be processed, regardless if one of them fails.
- Use when you want to see the end result of all elements passed.
For Each vs. Parallel For Each
(Image courtesy of Salesforce Ben)
Summary
Both For Each loop connectors are great tools for building a MuleSoft API. Each project will have different requirements. Based on your project requirements, you will now be able to figure out which connector to use.
Ad Victoriam’s certified, experienced and passionate MuleSoft consultants team up with businesses to turn legacy point-to-point integrations into a scalable architecture that your business can grow with. Let’s talk about a solution for your company today.
Related Resources:
Set Up Your Salesforce Org for Integration with MuleSoft
Subscribe to the AdVic Salesforce Blog on Feedly:
