Without knowledge what Talend context variables are and how to use them, you’ll never be able to take full advantage of this tool. And even if you workaround them, context variables would do the same a few times faster and simpler.
In this training:
- I will explain what are context variables
- I’ll tell you why you need them
- and I’ll show you examples of how you can use them to make your work easier.
Talend context variables
Context variables are primarily values assigned under a certain name that can change during program or process operation. Imagine the situation that you created a process that loads data from a file into a database. So you had to include in the source component path to the file, its name, and file separator and the target component include username and password to database. These values sometimes change, especially the password. What should you do then? Will you edit your process during each change? No, this is not feasible. This is where context variables are handy.
In Talend Studio you can create individual context variables and context groups. You can use context groups in many ways, e.g. to:
- isolation variables for DEV, TEST or PROD environments
- grouping configuration of a specific database (server name, user name, password, etc. in one group)
- grouping of related variables, e.g. paths to configuration files.
Therefore, context variables allow you to use different variable values in different environments, which allows you to quickly test the process in any environment you choose.
Adding context variables to the process
Metadata as a group
For this training, I created a simple process that loads data from a file into a MySQL database using record filtering in tFilterRow. As you can see in the screen below, the path to the file and its name is hardcoded, which in the case of changing the path or file name can exposes errors and force editing the entire process.
There are two ways to add a context group:
- Repository -> Contexts -> Create context group (unfortunately we would have to change the Property Type from Repository to Built-in later)
- Editing the schema in Repository -> Metadata
The second way allows us to save the schema as a Repository, so we’ll use it in the instructions below.
To edit the metadata for the customer_data source, all you have to do is click on any field in the Component tab, e.g. File name / Stream. A pop-up will appear with the option of changing the settings to a buit-in or updating the connection. So click OK.
In step 3/3, select Export as context. A pop-up will appear in which we want to create a new context group. I called my group C_10_SRC_Customer.
Talend Studio will automatically add values to the context group. To finish creating the group, select Finish. The studio will also ask about the propagation of changes in all processes which are using the source file – you should agree.
You should see your context group in the Contexts tab, and in the Component tab all file settings should be preceded by the word context, e.g. context.customer_data_File.
Single variables in context tab
There is a reason that tFilterRow is in the process, we will want to put a variable into the context tab which is used for filtering the data. To do this, add a new variable of type INT in the Context tab. My variable is called V_age_filter and has a value 30.
Now go to the Component tab and enter the name of your variable in the Value field, prefixing it with the word “context”.
Separation of environments using contexts
As I mentioned earlier, using context groups we can separate environments and run the same process in different environments with just one click. Talend creates all the variables you create in the Default environment. Let’s add a new environment called TEST.
To do this, in the Repository select Context -> <Your context group name> -> Edit context group. In step 2/2, select the green plus sign on the right side of the window.
Then add a new group by selecting New …, entering its name (TEST) and finally click OK. In the Context tab you can now see two groups: Default and Test and change them for the duration of the process.
If you enjoyed this post please add the comment below and share this post on your Facebook, Twitter, LinkedIn or another social media webpage.
Thanks in advanced!