topic Re: Newbie questions in Talend Studio

Newbie questions

_AnonymousUser — Sat, 16 Nov 2024 11:57:27 GMT

Hi,
I am evaluating Talend Open Studio for my company. We are mainly a Microsoft shop and would prefer not to code in Java. I realize this is a Java code generator, we are perfectly fine as long as we can stay within gui development/configuration. My first two newbie/dumb questions are:
1. What is the difference between TOS Data Integration and ESB? I downloaded both and at a glance, didn't notice any apparent differences between the two. We are interested in a service bus, so I've been playing with TOS ESB for a couple of weeks now.
2. In a simple job with tMOMOutput followed by tMSSqlOutput, how do I make both components run within the same transaction? I would like to see both database commit and message acknowledgement happen within one atomic transaction. Either both happen or neither, under any circumstances.
Thanks in advance!

Re: Newbie questions

Anonymous — Wed, 31 Jul 2013 03:28:57 GMT

Hi,

What is the difference between TOS Data Integration and ESB?

Only Talend ESB allows developers to develop, build, test and publish Java Web services, REST applications, data services and messaging routes ? all from a single user interface. http://www.talend.com/products/esb

In a simple job with tMOMOutput followed by tMSSqlOutput, how do I make both components run within the same transaction?

Do you want to get one output in Mssql DB and the other one is to ActiveMQ?
The workflow structure should be: input-->mian-->t<DB>output-->mian-->tMOMOutput
or input-->tReplicate-->mian-->t<DB>output
-->main-->tMOMOutput
Best regards
Sabrina

Re: Newbie questions

_AnonymousUser — Wed, 31 Jul 2013 13:53:02 GMT

Thanks Sabrina. Sorry, I've made a typo on my 2nd question above. The input is ActiveMQ and the output is SQL Server. So, my test flow looks like this:
tMOMInput --> tMap --> tMSSqlOutput
Can the ActiveMQ client acknowledgement and the database commit both happen within a single transaction? Is it possible that the server/process dies in between database commit and message acknowledgement so that a record is written to database but the message is not acknowledged? (<--this is undesirable for us)

Re: Newbie questions

Anonymous — Thu, 01 Aug 2013 08:30:59 GMT

Hi,

Can the ActiveMQ client acknowledgement and the database commit both happen within a single transaction?

It can be achieved in a single transaction.
Such as:
tRowrenerator--mian-->tMomoutput-->oncomponentok-->tMOMInput --> tMap --> tMomOutput-->tMysqlOutput.

Can the ActiveMQ client acknowledgement and the database commit both happen within a single transaction? Is it possible that the server/process dies in between database commit and message acknowledgement so that a record is written to database but the message is not acknowledged? (<--this is undesirable for us)

It is possible that a record is written to database but the message is not acknowledged on ActiveMQ when the server/process crash suddenly.

Best regards
Sabrina

Re: Newbie questions

_AnonymousUser — Thu, 01 Aug 2013 20:07:25 GMT

Thanks for the clarification Sabrina. Is there a way to implement this within Talend? I found some old posts here discussing about XA Transaction support. But those posts date back a couple of years. Has anything changed since then to support XA Trasnaction in Talend?

Re: Newbie questions

Anonymous — Tue, 06 Aug 2013 03:23:36 GMT

Hi,
Perhaps this Talend ESB Mediation Developer Guide: TalendESBMediationDeveloperGuide53EN/3.24.2 Notesfor XA Transaction part will be helpful for you.
Best regards
Sabrina

Re: Newbie questions

_AnonymousUser — Wed, 07 Aug 2013 15:15:51 GMT

Hi Sabrina,
That article seems to address messaging transaction only, it does not include database transaction. Please correct me if I am wrong.
In my example, for message acknowledgement and database insert, is it possible to make both happen, or neither happen under any circumstances? This is what I meant by "XA transaction" or "atomic transaction" across both messaging and database. This makes it impossible to have a database insert committed but message not acknowledged. Either both happen or neither happen under any circumstances, such as sudden server crash or network issues.

Re: Newbie questions

_AnonymousUser — Mon, 12 Aug 2013 14:12:02 GMT

Hi again,
I found this article describing exactly what I was looking for (in the top section):
http://activemq.apache.org/should-i-use-xa.html
Is this supported by TOS ESB out of the box? Or do I have to write some Java code?

Re: Newbie questions

Anonymous — Tue, 13 Aug 2013 04:03:36 GMT

Hi Chung Chang,
Sorry for the delay, so far talend don't support for XA transactions (two phase commit?) as the article mentioned both in Talend Open Studio for ESB and Talend Enterprise Subscription Version.
It is welcome to open a JIRA issue in the Talend ESB project of the JIRA bugtracker for a new feature.
Best regards
Sabrina

Re: Newbie questions

_AnonymousUser — Tue, 13 Aug 2013 14:39:36 GMT

Thanks again Sabrina. I have opened a new issue as you have suggested: https://jira.talendforge.org/browse/TESB-10795#. I am surprised that this feature isn't more sought after. No infrastructure is 100% reliable. So, how do your users typically deal with inconsistent transaction states after an unexpected infrastructure crash?

Re: Newbie questions

Anonymous — Wed, 14 Aug 2013 04:03:23 GMT

Hi,
Thanks for your feedback and your suggestion.
We plan to do XA transactions in the further job. So far, you can custom it using sprint.
Best regards
Sabrina

Re: Newbie questions

_AnonymousUser — Wed, 21 May 2014 06:05:52 GMT

Hi Sabrina,
I new to the Talend ETL tool i have a requirement as below
1. I want to do it a customized logging for all transactions which ever get fails.
2. When i will run job if founds any DB exception then its going to dis connect and not handling Exception/Error
I want to handle it
Please help me out i need your suggestion ASAP, much appreciated.

Re: Newbie questions

Anonymous — Wed, 21 May 2014 07:00:13 GMT

Hi RizwanSharief,
Would you mind opening a new topic for your requirement and set an example with real working scenario in it.
Best regards
Sabrina

Re: Newbie questions

Anonymous — Wed, 21 May 2014 17:07:20 GMT

Can you open the new topic in the following forum: https://community.talend.com/t5/Archive/Sybase-Connection/td-p/56802
which is dedicated to ETL (data integration) rather than the ESB forum.
Many thanks