topic Re: Assign Context Variable from a DB fetch in Talend Studio

Assign Context Variable from a DB fetch

tnewbie — Sat, 16 Nov 2024 05:24:07 GMT

Hello All,

In an audit table that captures run statistics of all the jobs running in a batch, i want to populate a column BatchID and i want this value to be a incremental running sequence. For this, every time i run the job i want to fetch the max value of the existing BatchID from this table and +1 and populate that value (i don't want to use DB identity property for this for all the known reasons).

My thought process is, have a context variable nxt_batch_id (integer) and have the Implicit contextLoad enabled at the job level, so that I can write the query select max(batchID) from audit_table.

I am not sure if i am conceptually doing anything wrong...can you please let me know if i am moving away from the way it needs to be done or where am i going wrong. As i am not great at Java trying to achieve this through DB queries vs writing custom code.

Please see the attachments

Re: Assign Context Variable from a DB fetch

tnewbie — Wed, 03 Jul 2019 14:39:57 GMT

Hello,

Can someone please help with the request "Assign Context Variable from a DB fetch"...

Simply put all i want to know is, how to fetch a value from DB and pass it over to a parameter in TalenD and use that parameter in the process flow.

This is certainly a kids play in all other popular tools i have worked with, but being new to TalenD and non-java resource and owing to tool complexities, i am seeking help on this.

Thanks

Re: Assign Context Variable from a DB fetch

Anonymous — Wed, 03 Jul 2019 15:11:54 GMT

Hi,

If you want to fetch a data from DB and assign it to context variable, it pretty simple in Talend too.

If you want to pass to another job, that is also pretty easy. You can use a tRunJob component to call another job and select the option to pass the context variables.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Assign Context Variable from a DB fetch

tnewbie — Wed, 03 Jul 2019 15:29:24 GMT

Thanks Nikhil...for your response

I guess your approach holds good if i have a tjava component...In my case where i am fetching a seq. number from a table and use that parameter/context value across the board, I guess using job level select approach is appropriate...can u pls check the attachment...for ur quick review i have marked areas to be looked into in red

PleaseNote: I am trying to avoid a dummy pipeline like tdbinput --> tjavaRow because i am trying to build a reusable component and if i go the tjava route, i will end doing it in all the 100+jobs which I guess is a avoidable overhead

param value from db fetch.png

Re: Assign Context Variable from a DB fetch

tnewbie — Wed, 03 Jul 2019 17:31:37 GMT

Hello,

Can anyone please look into this issue...

Thanks

Re: Assign Context Variable from a DB fetch

Anonymous — Thu, 04 Jul 2019 02:18:00 GMT

Hi,

If you want to create a reusable way of capturing the job execution statistics, why are you creating a new set of job design? You can very well use the Talend Activity monitoring Console (AMC) for this activity.

https://talendpnp.github.io/architecture-center/reference-architecture/summer-2018-7.0/talend-components/talend-activity-monitoring-console

https://www.talend.com/resources/data-integration-amc-studio-features/

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂