Robert van Boesschoten
Last updated: 27-08-2020
When building an application, logic and workflows can make up a big part of it. This is done through Actions in Betty Blocks. To keep your application clear, avoid creating long actions, but instead, create multiple short actions that form a chain of actions. Especially when you're expanding your application, short actions are easier to maintain.
Another benefit of short actions is that you can call upon them in great amounts and still maintain good performance in your application. This on the contrary to huge actions processing huge amounts of data. Let's talk about building some efficient actions by using batches.
For example, you're processing a collection containing 10000 records. You could loop through all 10000 records and do what you wanted to do, but this is not efficient and could result in a timeout as it processes in serial. Instead, it's better to chop the collection into multiple, smaller collections of 100 records and pass them on to actions for parallel processing.
Below is a step-by-step plan on which we'll zoom in on afterward:
1. Create an action called Part 1 containing
ceil(var:total_amount / 100)
2. Create another action called Part 2 which is called upon in each loop iteration containing:
3. Create an action event in the Loop flow of Part 1, calling upon Part 2
(var:batch_number - 1) * int(var:limit)
4. Process a smaller, more manageable collection in Part 2 for each iteration of your Loop event, based on the values passed along from action Part 1.
Some of these steps may seem a bit vague, but we'll discuss each step accompanied with images to clarify.
The action where it all starts. This action would originally contain a collection variable too big to process efficiently. Instead, limit the variable to 1 and call it limited_collection.
With this collection, you'll create the collection you would normally use, but just with 1 record. Now we'll use this variable in another variable called total_amount. This time of the type number expression.
By doing this, you'll get the number of records in the collection, without rendering the whole collection. This is important for maintaining good performance.Now we know how many records we have, we can calculate the number of batches to process.
In this tutorial, we're working with a batch size of 100. Feel free to change it for your application, but stick to the same amount for the whole process. Create a number expression variable called number_of_batches to calculate the number of batches.
ceil(var:total_amount / 100)
After creating the action, add a Loop event to the action.
In the Loop event, we're not looping through a collection, but instead, we're looping the number of batches. Do this by selecting the number_of_batches in the Times option. Set the Index option to batch_number. This will make the current iteration available through a number variable called batch_number.
Create another action called Part 2 which is called upon in each loop iteration. Create 2 input variables: limit and offset.
These input variables will receive values from action Part 1, which we're going to use to define a new collection variable.
The Limit option on a collection variable makes sure no more records than the value the option holds are included in the collection. The Offset option on a collection variable makes sure the collection starts collecting records from the position equal to value the option holds. For now, we'll leave this action for what it is. First, we need to connect this new action to the action we made in the beginning: Part 1.
Now we're adding an action event within the Loop flow of action Part 1. In this action event we need to create 2 variables to pass on to the input variables from action Part 2: limit and offset.
This variable has the same value as our batch size: 100
(var:batch_number- 1) * int(var:limit)
With the offset variable, we can calculate which subset of records we are going to collect in the subaction of each iteration.
These variables can now be passed on to the input variables of Part 2.
This is the part where we're going to place the events we were originally going to execute in the 'inefficient' action. We determined how many records we're processing, the amount of batches it takes to process them in this subaction and designed a construction to pass on the values needed to collect the desired records.
For this example, we're going to loop through a collection of 10000 records, update a property in each record and send an email.