topic Re: Retain value of sequence generator in Talend Studio

Retain value of sequence generator

Anonymous — Wed, 16 May 2018 00:03:49 GMT

Hi All,

I want to have a sequence for one of the column and it should start the sequence from where the last run stopped.

for example if 10 was generated by sequence for first run , for second run it should start with 11.

I got the first half , to generate the sequence using numeric.sequence("s1",1,1)

can you suggest how can i implement the second part , i went through the forum but couldn't find the solution

Thanks in advance,

Prabhat

Re: Retain value of sequence generator

TRF — Wed, 16 May 2018 05:21:14 GMT

Store the value of the current run into (1) a file or (2) a database table or (3) a context variable and recreate your context file.
Then when the next run starts, get the value and push it to a global variable (for 1 and 2) and use it to set the sequence value like this: numeric.sequence("s1", (Integer)globalMap.get("yourVariable ", 1) or numeric.sequence("s1", context.yourVariable, 1) in the case of a context variable.

Re: Retain value of sequence generator

manodwhb — Wed, 16 May 2018 05:55:12 GMT

@Prabhat,please find the below way.

Re: Retain value of sequence generator

TRF — Wed, 16 May 2018 08:52:49 GMT

This is just a kind of visual representation of what I wrote before.

Nothing really new in that.

Re: Retain value of sequence generator

Anonymous — Wed, 16 May 2018 18:22:03 GMT

Thanks TRF for your reply.

I am not looking to store the values in file or database. Can you please tell me more about the 3rd way where we are storing the current value in context variable.

Thanks,

Prabhat

Re: Retain value of sequence generator

manodwhb — Thu, 17 May 2018 07:50:55 GMT

@Prabhat,without loading to file or DB it is not possiable as per my understanding.

@TRF,can you explain the way without loading into file or DB?

Re: Retain value of sequence generator

TRF — Sat, 19 May 2018 13:43:07 GMT

@Prabhat, sorry for the delay, here the full solution using a context file.

For the sample, the job design looks like this:

How it works:

- the context is stored in a properties file outside from the job package and loaded at runtime using the implicit tContextLoad functionality (Job properties, Extra, untick the "Use project properties" option, tick "Implicit tContextLoad, choose "From the file" and enter your filename which is "C:/Users/offic/Desktop/TestTalend/contextDump.csv" in my case, choose ";" or any thing else as the field separator)

- here is what my contextDump.csv file looks like:

hasmore;
sample;100
test;0

- the context variable used to store the current value is called "sample" with an initial value of 0

- when the job starts, the tJava component print its value on the console (see the end)

- then we go to the tFixedFlowInput component which is used to compute the new value for the sample context variable. As the variable must be a String for the nex component (tContextLoad), we have a double conversion to convert the value to an Integer, add 100 to the current value and convert the result back to a String:

- tContextLoad is used to modify the value of the context variables

- tContextDump / tFileOutputDelimited write these variables with their value into the context file used when the job starts (C:/Users/offic/Desktop/TestTalend/contextDump.csv)

- finally, tJava print its new value on the console:

Démarrage du job test a 14:20 19/05/2018.
[statistics] connecting to socket on port 3884
[statistics] connected
Initial value:100
[statistics] disconnected
Job test terminé à 14:20 19/05/2018. [Code sortie=0]

Each time the job starts, the value from the previous run is taken from the context file.

That's all.

Re: Retain value of sequence generator

TRF — Mon, 21 May 2018 09:33:44 GMT

@Prabhat, did this help you?

If so, thank's to mark this case as solved (Kudos also accepted).