topic Re: How to store Variable while running Talend Job in Talend Studio

How to store Variable while running Talend Job

_AnonymousUser — Fri, 13 Mar 2015 08:01:11 GMT

hi,
Scenario->
I want to store Some record in File or table with ID column.
Suppose, For first run Storing 10 record
then, O/P will be
ID
-----
1
2
3
.
.
10
Now for Second run it should start from 11. for that we need to store max of last load i.e10
so it will be
ID
---
1
2
3
.
.
10
11
12
13
...
how to do this in Talend.

Re: How to store Variable while running Talend Job

Anonymous — Fri, 13 Mar 2015 10:55:39 GMT

Hi
In the beginning of job, read the max ID from a static file and store it to a context variable or global variable for used later. For example:
tFileInputDelimimitecd--main--tJavaRow
on tJavaRow:
context.max_value=input_row.max_value;

In the main processing of job,
if source data are stored in database, select records only when ids bigger than max value, for example:
"select * from tableName where id>"+context.max_value
if source data are read from file or other data sources, filter the rows on tMap, and then load the result to destination. For example:
tFileInputDelimited--main--tMap--o/p component--main--tSortRow--tMap_2--main--tFleileOutDlimited_2
on tSortRow, sort all the ids with desc model.
on tMap_2: add a sequence number for each row, and filter the rows to get only the first row which is the max ID for this execution.
on tFleileOutDlimited_2: update the max ID to the static file.
Best regards
Shong

Re: How to store Variable while running Talend Job

_AnonymousUser — Fri, 13 Mar 2015 14:27:57 GMT

Hi Shong,
Thanks for reply. I did the same but I am getting same value again not newly max value.
Regards,
Kiran B.

Re: How to store Variable while running Talend Job

Anonymous — Sun, 15 Mar 2015 04:35:17 GMT

Hi
I don't know which step your are doing some things wrong, can you please export your job and send it to me?
BR
Shong