Could You Please Share This Post? I Appreciate It And Thank YOU! :) Have A Nice Day!

Imagine that you are an employee of a large corporation and your task is to build dozens of data migration processes. [ Talend Master and Stadalone job] You have built them all, performed unit tests of individual migration tasks and would like to finally free yourself from running all these processes separately. There are several possibilities – a script with running all processes in the right order, a tool for automating processes or simply nesting all processes into one in Talend DI.

In this lesson I will try to explain what master and standalone processes are, how to build them and why you should use them.

What is a Talend Master and Stadalone job?

The master process in Talend Data Integration is a set of subprocesses in one integration task run in a specific order. All subprocesses are linked together with OnSubjobOK links. However, we must not forget that when using the OnSubjobOK connection, we should also specify the OnSubjobError path.

You can only run all processes built in Talend DI in a dedicated tool. Unless … you export them as independent, i.e. standalone processes. Talend gives you the opportunity to export processes as Unix files (sh script) or Windows (bat format script).

Talend DI Tutorial: Talend Master and Stadalone job - check on 1 easy example!

Building a master process

Create a new process in the repository and drag into the Designer workspace processes that you want to run in one job. Then connect them using the OnSubjobOK link. (Talend Master and Stadalone job)

You can add processes using the empty tRunJob component and supplementing such attributes as the scheme and name of the process or drag and drop them from Repository manager.

Then, from each process, follow the OnSubjobError link to tDie components – if you want to stop the master process if one of the internal processes fails.

Talend DI Tutorial: Talend Master and Stadalone job - check on 1 easy example!

Your first master process is ready – try to start it. If all subprocesses have been successfully performed, the master job will execute correctly and return a ‘0’ completion code indicating success. (Talend Master and Stadalone job)

Make sure that your tRunJob component has the “Die on child error” box checked, which allows you to stop the process on the failed subprocess. Thus, if this option is not selected – the process will perform further subprocesses despite the errors encountered.

Talend DI Tutorial: Talend Master and Stadalone job - check on 1 easy example!

However, if any of the subprocesses fails, the master process will fail and stop loading further data. (Talend Master and Stadalone job)

Talend DI Tutorial: Talend Master and Stadalone job - check on 1 easy example!

Standalone job

To create a standalone process:

  1. Select a process in the repository, right-click on it and select Build Job
  2. Choose interesting parameters – especially pay attention to Shell launcher (All / Windows / Unix) and Context scripts, where you can choose the context in which you want to create the master process
  3. Click Finish.
Talend DI Tutorial: Talend Master and Stadalone job - check on 1 easy example!

In the exported folder you will find processes in jar format, bat process for the Windows operating system, and if you have also selected the Unix option – sh script. Scripts for individual operating systems are commands that run your subprocesses exported in jar format in the order specified in Talend DI.

Could You Please Share This Post? 
I appreciate It And Thank YOU! :)
Have A Nice Day!

BigData-ETL: image 7YOU MIGHT ALSO LIKE

How useful was this post?

Click on a star to rate it!

Average rating 4.9 / 5. Vote count: 88

No votes so far! Be the first to rate this post.

As you found this post useful...

Follow us on social media!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments