Parallel Task Orchestration with ELSA 3.0
By Sipke Schoorstra · 2024-01-07

Simplifying Concurrent Tasks Photo by fabio on Unsplash · Parallel Task Execution · The Elsa Way ∘ Visual Representation ∘ Sequential Invocation, Concurrent Execution ∘ Achieving True...
Simplifying Concurrent Tasks
· Parallel Task Execution
· The Elsa Way
∘ Visual Representation
∘ Sequential Invocation, Concurrent Execution
∘ Achieving True Concurrency
∘ The Impact of Asynchronous Execution
· Seeing it in Action
· Other Jobs
· Horizontal Scalability
· Summary
Parallel Task Execution
In the world of software development, we often find ourselves needing to execute multiple tasks simultaneously, and then handle the outcomes once all tasks are complete.
Take a scenario where you need to make three HTTP calls at the same time. In C#, this can be achieved neatly and efficiently. Here’s how you might approach it:
using System.Diagnostics;
// Setup an HttpClient.
var httpClient = new HttpClient();
// Create a list of tasks.
var httpRequestTask1 = httpClient.GetAsync("https://reqres.in/api/users/1?delay=3");
var httpRequestTask2 = httpClient.GetAsync("https://reqres.in/api/users/2?delay=3");
var httpRequestTask3 = httpClient.GetAsync("https://reqres.in/api/users/3?delay=3");
// Begin timing.
var stopwatch = new Stopwatch();
stopwatch.Start();
// Execute all requests in parallel.
var responses = await Task.WhenAll(httpRequestTask1, httpRequestTask2, httpRequestTask3);
// Print the responses.
foreach (var response in responses)
Console.WriteLine(quot;{response.StatusCode} response from {response.RequestMessage!.RequestUri}:");
// Stop timing.
stopwatch.Stop();
// Print the elapsed time.
Console.WriteLine(quot;Finished in {stopwatch.Elapsed.TotalSeconds} seconds");
The beauty of this program lies in its parallel execution. In this example, a single HTTP request takes 3 seconds. Running them sequentially would take 9 seconds for the program to complete. However, executing them concurrently, as shown, cuts the total time to just 3 seconds.
The Elsa Way
Transitioning from C# to Elsa workflows opens up new possibilities in handling parallel tasks, simplifying parallel task orchestration visually. Let’s explore how we can replicate the execution of 3 HTTP requests in parallel, as we did in C#, but this time within an Elsa workflow.
Visual Representation
The following workflow demonstrates the execution of 3 HTTP requests in parallel:

Sequential Invocation, Concurrent Execution
Although it may appear that Elsa executes these activities in parallel, the engine technically invokes them sequentially. The real magic of concurrency in Elsa lies in how the activities are processed.
Achieving True Concurrency
True concurrency in Elsa is achieved by allowing activities to perform work in the background. Here’s how it’s done:
1. Asynchronous Configuration of Activities: In the case of an HTTP Request activity, you can set it to run asynchronously. This is visually represented in Elsa by toggling the “Run Asynchronously” option:

2. Sequential Scheduling, Concurrent Execution: When an activity like an HTTP Request is encountered in the workflow, it’s scheduled for execution as usual. However, if “Run Asynchronously” is checked, the engine does something clever:
- It creates a background job and associates it with a bookmark.
- This process is quick, allowing the engine to move on and schedule the next activity, which might also be set to run asynchronously.
2. Concurrent Jobs: In our scenario, this means three jobs are scheduled to run concurrently.
3. Completion and Bookmark Resumption: Once a job completes, it resumes the workflow using its bookmark, effectively synchronising the parallel processes.
The Impact of Asynchronous Execution
Without the “Run Asynchronously” setting, the entire workflow would take roughly 9 seconds to complete, assuming each HTTP request takes 3 seconds. However, with asynchronous execution enabled, the workflow can complete in about 3 seconds. This demonstrates the power of Elsa in handling parallel tasks efficiently, mimicking the concurrency we achieved in C# but in a workflow-based environment.
Seeing it in Action
One of the best ways to appreciate the power of parallel task execution in Elsa is to witness it firsthand. Follow these steps to set up an environment where you can see the significant difference that asynchronous execution makes.
- Get the Workflow File: Download this workflow file.
- Set Up Elsa Server + Studio: Fetch the latest version of Elsa Server + Studio Docker container and run it with the following command:
docker pull elsaworkflows/elsa-server-and-studio-v3:latest
docker run -t -i -e ASPNETCORE_ENVIRONMENT='Development' -e HTTP_PORTS=8080 -e HTTP__BASEURL=http://localhost:13000 -p 13000:8080 elsaworkflows/elsa-server-and-studio-v3:latest
3. Access Elsa Studio: Navigate to http://localhost:13000/
4. Log In: Use the default credentials:
username: admin
password: password
5. Import the Workflow:
- Select Workflows -> Definitions from the menu
- Import the file you downloaded in step 1.
6. Open and Execute the Workflow:
- Select the imported workflow to open it in the workflow designer.
- Click the green play button to execute the workflow. Observe that it completes in approximately 3 seconds.
7. Modify and Re-Test:
- Update each HTTP Request activity by selecting them and unchecking the “Run Asynchronously” setting.
- Re-execute the workflow as in step 6. You’ll notice a stark difference: the execution time jumps to around 9 seconds.
Other Jobs
Expanding the Scope of Concurrent Execution in Elsa
Elsa workflows aren’t limited to just executing HTTP requests in parallel. The frameworks versatility extends to a wide range of activities, and it even supports custom activities with background execution. This capability significantly broadens the potential applications of Elsa in various scenarios.
Writing your own activity that supports background execution is straightforward in Elsa. This can be done by specifying the kind of activity as either Task or Job.
For example:
[Activity("MyNameSpace", "MyCategory", "My description", Kind = ActivityKind.Task)]
public class MyCustomTaskActivity : CodeActivity
{
}
[Activity("MyNameSpace", "MyCategory", "My description", Kind = ActivityKind.Job)]
public class MyCustomJobActivity : CodeActivity
{
}The key distinction lies in their execution control:
- Task Kind: Offers the user control over whether to run the activity in the background.
- Job Kind: Automatically runs in the background, without user intervention.
By harnessing the power of Tasks and Jobs, workflows can be tailored to a wide variety of parallel processing needs, from simple HTTP requests to complex, custom-defined operations.
Horizontal Scalability
A key difference between using Elsa concurrent activity execution and the C# example earlier, is the approach to task dispatching. Elsa leverages a queue-based system, which can range from simple in-memory queues for less complex tasks to robust service bus queues like Azure Service Bus and RabbitMQ for larger-scale applications. This method contrasts sharply with the direct execution approach seen in C#, where all three tasks are executing in-memory in the same application instance.
The key advantage of this system is its ability to scale out horizontally. As workload demands increase, more resources or nodes can be added, significantly enhancing the system’s capacity to handle a higher volume of workflows efficiently. This approach not only ensures a more robust and reliable execution of tasks but also caters to scalable solutions, making Elsa a compelling framework for optimising workflow throughput in diverse operational scales and complexities.
Summary
Elsa 3’s parallel task execution capabilities significantly reduces overall execution time when there is a need for running & orchestrating multiple jobs.
Key Highlights:
- Parallel Execution in C#: We began by illustrating how multiple HTTP requests can be executed concurrently in C#, a method that significantly cuts down on execution time compared to sequential processing.
- The Elsa Workflow: Transitioning from C# to Elsa, we demonstrated how similar parallel execution can be achieved in workflow environments. By leveraging the “Run Asynchronously” setting in Elsa, tasks that would normally take longer to complete sequentially are executed in a fraction of the time.
- Beyond HTTP Requests: Elsa’s capabilities are not limited to HTTP requests. We showed how both built-in and custom activities can be designed to run as background tasks or jobs, thus broadening the spectrum of applications.
Through this exploration, I hope to have shed light on the versatility and power of Elsa 3.0, opening up avenues for innovative workflow solutions across various industries and applications.