topic Re: Embedded Talend? in Talend Studio

Embedded Talend?

Anonymous — Sat, 16 Nov 2024 14:19:39 GMT

Is it possible to use Talend embedded inside another application (similar to how CloverETL allows embedded mode)? The goal would be to programmatically generate an ETL pipeline and then have Talend run it.
Thanks!

Re: Embedded Talend?

Anonymous — Mon, 23 Jun 2008 06:24:04 GMT

Hello
tSystem component executes one or more command as you do on CMD. This component can typically used for companies which already implemented other application that they integrate into their processing flow through Talend.
Best regards
shong

Re: Embedded Talend?

Anonymous — Tue, 24 Jun 2008 14:49:51 GMT

Instead of having Talend call another process using tSystem, can the reverse be done? Can I have my system configure and run a Talend pipeline - so Talend is embedded into my application?
Thanks!

Re: Embedded Talend?

fyk_cz — Tue, 24 Jun 2008 15:02:53 GMT

You can export your job as a Java POJO package (or as an Axis WebService). I'm using this packages for my projects (in java) everyday.

Re: Embedded Talend?

Anonymous — Tue, 24 Jun 2008 16:17:21 GMT

Exporting as a Java POJO sounds great for running the job, but doesn't that still require the UI to output the POJO?
Is there a way to have Talend programmatically generate the ETL pipeline, and then generate the runtime Java POJO? For example, suppose I know I have two sources of data and want to join them and output them to a file. Does Talend have an API for creating a new ETL pipeline, configuring the two sources and sink, adding the join, and exporting the pipeline - all without requiring user interaction?
Thanks!

Re: Embedded Talend?

Anonymous — Tue, 24 Jun 2008 16:47:19 GMT

Hi,
Talend Export Job Scripts feature allows you to deploy and execute a job on any server, regardless
Talend Open Studio.
The job scripts export adds to an archive all the files required to execute the job, including the .bat and
.sh along with the possible context-parameter files or relative files.
The export type include also Java POJO, Axis Webservice (WAR) and Axis,Webservice (Zip).
Best look

Re: Embedded Talend?

Anonymous — Tue, 24 Jun 2008 22:44:19 GMT

Perhaps my question is not clear. I understand that I can export an ETL pipeline I've created in the UI and then run that job embedded in another Java application.
My question:
Can I create a new job, configure it, and export it using *only* the Talend API, not using the Talend UI at all?

Re: Embedded Talend?

fyk_cz — Tue, 24 Jun 2008 22:54:21 GMT

Is there a way to have Talend programmatically generate the ETL pipeline, and then generate the runtime Java POJO? For example, suppose I know I have two sources of data and want to join them and output them to a file. Does Talend have an API for creating a new ETL pipeline, configuring the two sources and sink, adding the join, and exporting the pipeline - all without requiring user interaction?

I mean you need own task generator, for recompute all in the space ...
Create a job for a new data transfer, export the job to POJO package and embede generated package to your own app. If you need transfer something from somewhere to someplace, you need "somegenerator". But I mean, with well configured and prepared Talend job, you can use only one package and "something" define via properties and contexts.

Re: Embedded Talend?

Anonymous — Wed, 07 Oct 2009 23:16:20 GMT

mikem2005, did you ever get your question answered? I'm looking for the same thing. As far as I can tell the only way to create a new job is through the TalenD UI.
There doesn't seem to be an API exposed for other applications to create jobs.

Re: Embedded Talend?

Anonymous — Thu, 08 Oct 2009 05:39:24 GMT

Looks like Pentaho kettle uses an API. May please try to find out .
Please let us know in case it is useful.
Thanks and Regards,
Pravu Mishra.

Re: Embedded Talend?

Anonymous — Thu, 08 Oct 2009 21:14:46 GMT

Well, TOS is opensource, it'll be a pain, but you can try to look at the jar files & their API.
A talend job is basically a xml file (see .item files) serialized from an EMF model (see the eclipse project).
TIS (the non free version of TOS) has a console which can then compile the project.

Re: Embedded Talend?

Anonymous — Fri, 09 Oct 2009 05:18:40 GMT

Hi Morbo,
The question is not about design of Talend and using Eclipse modelling framework(EMF) or java classes without having intergace implementation.
The question is about whether there is/are APIs which can be used independently from out side of TOS. Please let us know in case you have some idea about the API/s exposed by talend.
There are many third party APIs those are used by talend. Might be there are APIs developed by talend on top of third party API. Do you have any idea about it?
Any way talend is an excellent software.
Thanks and Regards,
Pravu Mishra.

Re: Embedded Talend?

Anonymous — Fri, 09 Oct 2009 17:24:37 GMT

Hello,
I don't know much, I just did some digging on some area.
All I'm saying is that's it's already possible to manipulate the .item file in a regulated manner (thanks to EMF) and without TOS. I'm made a basic gui launcher wich uses this to read the item file, deduce the context parameters to expose.
See this about the EMF model of talend https://community.talend.com/t5/Design-and-Development/Is-there-a-the-xsd-for-item-files/td-p/99896
And this was a start of discution about talend use of GEF : https://community.talend.com/t5/Design-and-Development/Which-class-of-TOS-generates-the-GUI-representation-of-a-job/td-p/102934 but I didn't go further.
Feel free to share your investigations .
Regards

Re: Embedded Talend?

Anonymous — Wed, 30 Mar 2016 15:50:13 GMT

Hello,
I know this post has been asleep for a while but the question has never been properly answered.
So, once again, is there today a way to define programmatically talend Job and execute them from my own java application ?
I checked the documentation, there is a guide for components development but that is not what i want, there is (to my knowledge), there is no available API, nothing useful on the maven repositories so...
Did i miss something ? Is it even possible to do what i'm asking ? I saw some tutorials about running already defined Talend Job from java application, i can maybe turn aroud this to build my own application but i'd like to be sure...
Sorry for the inconvenience,
Best Regards, Cyprien.