topic Re: Bug while generating a Surrogate key for Incremental Load. in Talend Studio

Bug while generating a Surrogate key for Incremental Load.

Anonymous — Sat, 16 Nov 2024 05:16:14 GMT

Hi,

I have a job in which i am loading two tables, in those tables i am generating Surrogate id,

The process which I am generating this is, before running the job I am taking the max value of that column into a context and using this context for incrementing by 1 for each row.

Here the bug which I found is that, Suppose I am loading the 1st job after that i am using onsubjobOk to start the second job.

in this case the context value of 1st job itself is taking by 2nd job as well, instead of taking its own context value.

Can anyone help me how to avoid this situation.

Quick help is appreciated.

Regards

ShaX

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Tue, 16 Jul 2019 14:47:51 GMT

Hi,

Could you please mark where you are updating the context variable in the attached screenshot? Component screenshots will be helpful to understand your flow better .

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Tue, 16 Jul 2019 15:43:03 GMT

Hi @nthampi ,

Thanks for the reply, PFA images below.

This is the jobHere am taking the max value of the column n storing it in the context.That context am using to populate the column.

Please help me on this.

Regards

ShaX

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 17 Jul 2019 03:11:39 GMT

Hi,

Since you are using assignment to context, could you please use tjavarow instead of tMap?

Please also print the context variable before and after to confirm that the value is changing using System.out.println(context.variable_name)

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 17 Jul 2019 12:58:46 GMT

Hi @nthampi ,

As told by you, instead of Tmap, i took Tjavarow to assign value to the context.

However this method also didn't work.

PFA screenshot for your reference.

Can you help me on this.

Regards

ShaX

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 17 Jul 2019 14:26:15 GMT

Hi,

Could you please share the component screenshot along with print output?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 17 Jul 2019 15:38:15 GMT

Hi @nthampi,

This is how I am loading the tables using contex.

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 17 Jul 2019 15:43:27 GMT

Hi,

That is fine. But where are you verifying the current value of the context variable?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Mon, 22 Jul 2019 06:49:08 GMT

Hi @nthampi ,

My issue is resolved,

what is did is, if i am loading 10 tables in 1 job, earlier i used to use "S1" in all table in numeric sequence, but now am using from "S1" to "S10" for tables respectively.

I don't know weather this is the correct approach, but it is working fine as of now.

Regards

Shax.

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 24 Jul 2019 07:44:34 GMT

Hi @nthampi ,

Am facing same issue again after 2 days of job run.

Can you help me what can be done.

Here you can see am printing the value.

The problem is coming while loading all tables in bulk.

Is there any other way to assign a context and use where i can avoid this issue.

Currently am assigning "context.TEST = row1.KEY" and using "Numeric.sequence("s2",context.TEST,1) "

regards

ShaX.

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 24 Jul 2019 13:37:49 GMT

Any help on this.

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 24 Jul 2019 14:55:41 GMT

Hi,

Could you please add screenshots of the latest job flow, your current sequence generation process and the stage where you are facing issue with some sample records?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 24 Jul 2019 18:01:58 GMT

Hi @nthampi ,

PFA screenshots for your reference.

The job:

Assigning the value to the context in Tmap2 in above picture.

Using Numaric.sequence and context for to load the table.

The problem is not coming while loading single table.

it is coming wile loading all tables in bulk.

Regards

ShaX

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 24 Jul 2019 18:07:23 GMT

Basically, the error am getting is

ORA-00001: unique constraint (XXXXXXXX.DW_DIM_LOAN_MASTER_PK) violated.

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 24 Jul 2019 18:45:05 GMT

Hi,

Could you please show me how you are allocating context variables using tMap? Ideally you should use a tContextLoad or tjavarow to assign the values to context variables.

Did you print and checked whether the assignment is correctly done?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 24 Jul 2019 19:25:10 GMT

I don't have screenshot of the output i printed, but i shared the screenshot in the above conservation.

It works.

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 24 Jul 2019 19:28:35 GMT

Hi,
Can you please mail me on this mail, so that i can share my job.
sharukh.talend@gmail.com

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 24 Jul 2019 19:37:53 GMT

Hi,

Unfortunately I will not be able to share my email id. If you are looking for a screen sharing session, I would recommend you to raise a support ticket and they will look into the issue.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 24 Jul 2019 19:49:37 GMT

No worries @nthampi ,

Can you tell me the process to raise the ticket.

Re: Bug while generating a Surrogate key for Incremental Load.

Anonymous — Wed, 24 Jul 2019 19:53:34 GMT

Am using Open studio,

Can I raise a ticket?