Imagine that you are an employee of a large corporation and your task is to build dozens of data migration processes. [ Talend" Master and Stadalone job] You have built them all, performed unit tests of individual migration tasks and would like to finally free yourself from running all these processes separately. There are several possibilities – a script with running all processes in the right order, a tool for automating processes or simply nesting all processes into one in Talend DI.

In this lesson I will try to explain what master and standalone processes are, how to build them and why you should use them.

What is a Talend Master and Stadalone job?

The master process in Talend Data Integration" is a set of subprocesses in one integration task run in a specific order. All subprocesses are linked together with OnSubjobOK links. However, we must not forget that when using the OnSubjobOK connection, we should also specify the OnSubjobError path.

You can only run all processes built in Talend" DI in a dedicated tool. Unless … you export them as independent, i.e. standalone processes. Talend" gives you the opportunity to export processes as Unix files (sh script) or Windows" (bat format script).

Building a master process

Create a new process in the repository and drag into the Designer workspace processes that you want to run in one job. Then connect them using the OnSubjobOK link. (Talend" Master and Stadalone job)

You can add processes using the empty tRunJob component and supplementing such attributes as the scheme and name of the process or drag and drop them from Repository manager.

Then, from each process, follow the OnSubjobError link to tDie components – if you want to stop the master process if one of the internal processes fails.

Your first master process is ready – try to start it. If all subprocesses have been successfully performed, the master job will execute correctly and return a ‘0’ completion code indicating success. (Talend" Master and Stadalone job)

Make sure that your tRunJob component has the “Die on child error” box checked, which allows you to stop the process on the failed subprocess. Thus, if this option is not selected – the process will perform further subprocesses despite the errors encountered.

However, if any of the subprocesses fails, the master process will fail and stop loading further data. (Talend" Master and Stadalone job)

Standalone job

To create a standalone process:

1. Select a process in the repository, right-click on it and select Build Job
2. Choose interesting parameters – especially pay attention to Shell" launcher (All / Windows" / Unix) and Context scripts, where you can choose the context in which you want to create the master process
3. Click Finish.

In the exported folder you will find processes in jar format, bat process for the Windows" operating system, and if you have also selected the Unix option – sh script. Scripts for individual operating systems are commands that run your subprocesses exported in jar format in the order specified in Talend" DI.

Could You Please Share This Post? 
I appreciate It And Thank YOU! :)
Have A Nice Day!