topic Re: How to create a generic job in talend in Talend Studio

How to create a generic job in talend

Anonymous — Thu, 04 Aug 2016 07:34:59 GMT

Hi Team,
Could you please confirm if we can create a generic job in talend which performs different operations on different sets of data.
Basically, we want to process multiple source files (having different schema) with different transformations but the overall processing flow in the job is same for all source files. (Eg a simple Job Flow : FileInputDelimited -> TMap -> tFileOutputDelimited).
If we can create such a generic job, then please help on below points :
1) How to save and pass different schema definition for different source files via parameter/file
2) How to pass key field via parameter/file
3) How to save and pass different transformations via parameter/file
Thanks
Ankit Gupta

Re: How to create a generic job in talend

Anonymous — Thu, 04 Aug 2016 09:14:41 GMT

Hi
It is impossible to do that, the schema must be defined before the job execution.
Regards
Shong

Re: How to create a generic job in talend

Anonymous — Tue, 07 Jan 2020 10:31:16 GMT

Hi,

This is a work around, which would work only for specific functions. Also, it will be tedious task to do so.

But you can create above given generic job.

Using tfileinput > Tjava > tfileoutput

In tfileinput use schema as dynamic.

In tjava ,use dynamic datatype and meta functions provided with dynamic type( like getColumnName, getColumnType ) to perform required functions.

In tfileoutput use dynamic schema to store file data.

Re: How to create a generic job in talend

Anonymous — Tue, 07 Jan 2020 14:51:12 GMT

Hi,

You might have to do custom coding but dynamic schema will help you to make it within a single job. A simple example can be referred from below Blog.

https://www.talend.com/blog/2019/10/31/magically-shuffle-columns-talend-dynamic-schema/

But you will have to make necessary changes based on your use case.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved