topic Re: Loop for job and output data usage in expression fields. in Talend Studio

Loop for job and output data usage in expression fields.

Anonymous — Mon, 24 Jul 2017 08:16:13 GMT

Hello,

I need to collect data from api.But its have row limit.I want to mean i can't collect all data in single operation.Its have pages and every page have row limit.I need to loop this job (totalpagecount) times for collect all data.

TWebservice > TNormalize > TMap > TFileOutputExcel

1. image : Current page value for input

2. image : Current page and total page values for output

Requirements:

My job need to run (Total Page Count) times.

currentpage value need to increase in every one loop

Loop need to be stop ^^

Example:

Input currentpage field will be start from 0.So its need to increase in every run.

Loop need to stop when currentpage = totalpagecount.

Loop run up to currentpage = totalpage

{

TWebservice > TNormalize > TMap > TFileOutputExcel

^<Tbufferoutput < (currentpage value???? with increasing every one loop)

}

How can i do it ?I hope for solution.

One more question :

how can i use mysql queries in expression fields ?I check this too much times but i can't found any topic.

Looking for help.

Thanks

Re: Loop for job and output data usage in expression fields.

Anonymous — Mon, 24 Jul 2017 08:26:39 GMT

Hi
First of all, get the value of total page and store it to a context variable, this variable will be the To value on tLoop component which is used to loop each page, eg:
twebserice1--main--tJavaRow
|
onsubjobok
|
tLoop1--iterate--tWebservice2--main....

twebservice1: get the value of total page if this value comes from a webservice.

twebservice2: pass the current page as input parameter with this expression:
((Integer)globalMap.get("tLoop_1_CURRENT_VALUE"))

Why do you ask 'use mysql queries'? What value do you need to execute a query to get?

Regards
Shong

Re: Loop for job and output data usage in expression fields.

Anonymous — Mon, 24 Jul 2017 09:44:32 GMT

Hello,

Thanks for this but i can't itirete tloop to twebservice.
Mysql queries for other jobs.Not important for now.

Re: Loop for job and output data usage in expression fields.

Anonymous — Mon, 24 Jul 2017 10:00:48 GMT

Hi burakbb
What's the issue? Why you said you can't iterate tloop to twebservice? Maybe I don't understand your problem well?

Re: Loop for job and output data usage in expression fields.

Anonymous — Mon, 24 Jul 2017 10:16:19 GMT

Hello,
Sorry i want to mean tloop can't be itirate to twebservice.
I am using open talend studio data integration.Could it be related to this?

Edit:

When i try to itirate them, its not complete.I can't use itirate feature on tloop to twebservice.Because its not allowed.Open talend studio data integration not allow this.

Re: Loop for job and output data usage in expression fields.

Anonymous — Tue, 25 Jul 2017 11:34:13 GMT

Hi
Just realize that twebservice don't accept an iterate link, need to redesign the job as below:
main job:
twebserice1--main--tJavaRow
|
onsubjobok
|
tLoop1--iterate--trunJob

trunJob: call the child job and pass the current loop value to child job. To do it, select the context variable which is defined in child job from the Context param table on tRunjob, and set its value as:
((Integer)globalMap.get("tLoop_1_CURRENT_VALUE"))

child job:
tWebservice2--main--tLogRow

Define a context variable let's called it current_value with int type in the Context viewer.

Regards
Shong

Re: Loop for job and output data usage in expression fields.

Anonymous — Tue, 25 Jul 2017 12:36:06 GMT

Hello Shong,

Ty so much.Already i fix that but its more better.I will apply your design.(I used uneffected component(Trest) for itirete them ^^)

But i am still not done.I need to put webservice_1 total page value to tloop_1 "To" field.I tried somethings but every time i got error.I can't set global variable in tjavarow.

Can you help me for this ?

tjavarow.png

Re: Loop for job and output data usage in expression fields.

Anonymous — Tue, 25 Jul 2017 16:01:30 GMT

What's the error message? To assign the page value to a context variable or global variable like this:
context.page_value=input_row.columnName;
//context variable page_value is defined as int type.
Then, you can use this variable as To field: context.page_value

Regards
Shong

Re: Loop for job and output data usage in expression fields.

Anonymous — Wed, 26 Jul 2017 03:04:26 GMT

Thank you so much.Its done Everything working correctly.

Re: Loop for job and output data usage in expression fields.

Anonymous — Wed, 26 Jul 2017 03:07:33 GMT

Great, thanks for your feedback!