Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in NYC Sept 4th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
talendstar
Creator
Creator

Dynamic Variable - iterate over list

I have a scenario where I need to iterate over a list(table column) and execute python code for whatever value variable holds at that moment and then move of the next value in the list - till all the items in the list have been processed.

 

Here is some background:

I have a table with a column that contains branch code.

 

TableA

BranchCode

123

124

125

126

127

128

129

130

 

Minimal Solution

What I would like my Talend flow is to iterate over the values in the column - for example for branch 123, set the variable, pass the variable to python code and execute python code. After that move on to next value in the column 124, set the variable, pass the variable to python and execute python code.. and so on.

 

 

 

Ideal Solution:

Ideally what I would like to do is split long list of values in the column between a parallelized workflow where each flow within the parallelized job take a set of values (let's say 25% each) and iterates over that. But this is more complex than minimal solution. So flow1 will get 123,124 and flow2 will get 125,125 and so on.

 

How can I accomplish this in Talend?

 

Thanks

Labels (3)
5 Replies
Anonymous
Not applicable

Hi @talendstar ,

 

You can use the TalendDBComponent ->tFlowToIterate->tRunJob (in tRunJob settings add the globalVaribale of flow to Iterate as context)

Your tRunJob will have the code for python.

 

Thanks and Regards,

Subhadip

talendstar
Creator
Creator
Author

Is it possible to pass the variable to a Python script that is run with tRunJob? or is there any other way to pass the variable to a python script?

 

thanks

Anonymous
Not applicable

Hi,

 

    You cannot run the Python code directly using Talend components. The trick is to call them using Talend terminal components like tSystem, tSSH etc. where you can pass the value to your Python script through Windows commands or Linux Shell Scripting.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

 

talendstar
Creator
Creator
Author

I usually run python scripts via .bat files and call the .bat file via tsystem.

 

Is there an example of passing variables to tsystem and then onto .bat file or python script that I can refer to?

 

thanks

Anonymous
Not applicable

Hi,

 

   Could you please check the below link? Its not exactly same but will give you an idea.

 

https://community.talend.com/t5/Design-and-Development/passing-convext-parameter-value-to-the-shell-...

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved