topic Re: [resolved] Generic Talend Job in Talend Studio

[resolved] Generic Talend Job

Anonymous — Wed, 09 Jan 2013 11:31:43 GMT

I want create a generic talend job with File Component , Tmap and Target Db Component.I have multiple files to load in multiple target tables with one talend job.But the metadata for each source and target are different.Let me know.

Re: [resolved] Generic Talend Job

Anonymous — Wed, 09 Jan 2013 11:41:49 GMT

For example:
for instance,
Table1.csv -> Generic_Talend_Job -> Oracle Table 1
Table2.csv -> Generic_Talend_Job -> Oracle Table 2
and so on...

Re: [resolved] Generic Talend Job

Anonymous — Thu, 10 Jan 2013 02:06:19 GMT

Hi
With community version, you have to create many subjobs for each file. With commercial subscription version, you are able to define a dynamic schema to map different structure, so that you only need one general Talend job for all files. For more information, please read this page:
https://help.talend.com/search/all?query=dynamic+schema+

Regards
Shong

Re: [resolved] Generic Talend Job

Anonymous — Thu, 10 Jan 2013 10:31:49 GMT

Thanks Shong

Re: [resolved] Generic Talend Job

Anonymous — Mon, 13 May 2013 15:01:14 GMT

Shong,
I have got the commercial version of talend.Used dynamic schema.It work very fine for the source files to db tables without any transformation logic.How can i achieve the adding transformation logic like nulls etc for individual columns in the Dynamic Schema?
Let me know.I have 10 sources file and 10 tables.Created one job to load this 10 tables using dynamic schema.But i want add different transformation logic for 10 sources.How to i achieve this?
Regards
Vasanth

Re: [resolved] Generic Talend Job

Anonymous — Mon, 08 Jul 2013 04:41:55 GMT

But i want add different transformation logic for 10 sources.How to i achieve this?

What's the transformation would you like to do for the column? Below is an example job shows you how to get the column Name, column value of the dynamic schema during the job execution, for example:
the table has the following structure and data:
id;name
1;shong
2;elise
begin code of tJavaFlex:

int ncount=0;
String columnName="";
DynamicMetadata column;

main code of tJavaFlex:

ncount=row7.c.getColumnCount();
column=row7.c.getColumnMetadata(ncount-1);
columnName=column.getName();
String columnValue=(String)row7.c.getColumnValue(columnName);
row7.c.setColumnValue(ncount-1, "Hello "+columnName);
row9.c=row7.c;

It generates a new file with the following result:
id;name
1;Hello shong
2;Hello elise
Shong

Re: [resolved] Generic Talend Job

Anonymous — Thu, 18 Jul 2013 21:20:53 GMT

Hi Shong,
I tried to use your code sample in a tJavaFlex, but I get a compile error telling me that DynamicMetadata cannot be resolved to a type. I see that we can import libraries directly into the tJavaFlex using its advanced settings, but I could not figure out which libraries to import before using this DynamicMetadata type. Can you or somebody else help me with this?
Thanks,
Douglas

begin code of tJavaFlex:
int ncount=0;
String columnName="";
DynamicMetadata column;
Shong

Re: [resolved] Generic Talend Job

alevy — Fri, 19 Jul 2013 06:20:36 GMT

Does you tJavaFlex have a row with a Dynamic column coming in or going out?

Re: [resolved] Generic Talend Job

Anonymous — Fri, 19 Jul 2013 14:11:49 GMT

Hi alevy,
Thank you for your answer!
I'm not using dynamic columns and now I realize that this example is specific for dynamic columns.

Douglas

Does you tJavaFlex have a row with a Dynamic column coming in or going out?

Re: [resolved] Generic Talend Job

Anonymous — Thu, 24 Jul 2014 12:01:50 GMT

After applying this solution. I figured that in Dynamic schema, the whole row is read as string with column value separated using comma(,). Please correct me if I am wrong.
Can we control which column is mapped to which column in the output component?
Please feel free to ask if any other details are needed to answer the question.

Re: [resolved] Generic Talend Job

Anonymous — Tue, 13 Jan 2015 00:17:09 GMT

Hi Shong,
I tried to use your code sample in a tJavaFlex, but I get a compile error telling me that DynamicMetadata cannot be resolved to a type. I see that we can import libraries directly into the tJavaFlex using its advanced settings, but I could not figure out which libraries to import before using this DynamicMetadata type. Can you or somebody else help me with this?
Thanks,
Douglas
What id c in the code? I am getting an error "c cannot be resolved"
begin code of tJavaFlex:
int ncount=0;
String columnName="";
DynamicMetadata column;
Shong

Re: [resolved] Generic Talend Job

Anonymous — Tue, 13 Jan 2015 00:18:38 GMT

What is c in the code? I am getting an error "c cannot be resolved"

ncount=row7.c.getColumnCount();
column=row7.c.getColumnMetadata(ncount-1);

Re: [resolved] Generic Talend Job

_AnonymousUser — Tue, 10 Feb 2015 09:32:42 GMT

c is the single column of type Dynamic, that must be created in the schema of MySQLInput component

Re: [resolved] Generic Talend Job

Anonymous — Sun, 19 Jul 2015 17:39:15 GMT

Hi Talend Experts, I am using talend enterprise version, I am aware of very basics of talend, I have requirement to create generic job, which takes source table name as input (I may setup this through context variable to pass table name) and generate csv files. I have lot many tables that I need to set up jobs to generate and send to Amazon S3. Please let me know if you have any demo examples that help me to proceed through.
I was looking through this thread found below link but it does not exist.
https://help.talend.com/pages/viewpage. … Id=5671283
Thanks, Pundalik

Re: [resolved] Generic Talend Job

Anonymous — Mon, 20 Jul 2015 04:22:46 GMT

Hi Pundalik
You can use tTableList to iterate all tables and pass the current table name to tMysqlInput (take mysql database for example in this case), define a Dynamic column on the schema, get each column name on a tJavaRow and se

Re: [resolved] Generic Talend Job

Anonymous — Thu, 06 Aug 2015 14:26:31 GMT

Well this is the common issue with the Talend architecture. It is fundamentally not designed for configuration based integration. Of course you are always free to customize it with your custom Java implementation to achieve such design. Still dynamic schema helped little bit, but it works of course only in the way where source and target structures are same.

Re: [resolved] Generic Talend Job

_AnonymousUser — Mon, 08 Feb 2016 08:04:39 GMT

Shong,
I have got the commercial version of talend.Used dynamic schema.It work very fine for the source files to db tables without any transformation logic.How can i achieve the adding transformation logic like nulls etc for individual columns in the Dynamic Schema?
Let me know.I have 10 sources file and 10 tables.Created one job to load this 10 tables using dynamic schema.But i want add different transformation logic for 10 sources.How to i achieve this?
Regards
Vasanth

Hi Vasanth,
Could you please share the generic Talend job details to load tables dynamically. I have same requirement in my project.
Regards,
Lavanya

Re: [resolved] Generic Talend Job

Anonymous — Tue, 08 Mar 2016 20:08:37 GMT

Hi All,
Can anybody tell me how to create dynamic schema using a text file where all the field names,datatype, length are present.
My requirement is
I have two files with different formats.For these two files i have to create a header and trailer for a set of records where header and trailer format is different for these two files. I have to create a generic module for header and trailer creation.
Hence i want to define the header structure of one file in one text file and the header structure of another file in another text file.
Using these two text files, during run time schema needs to be created in talend.
Is there any way? Please help me on this.

Re: [resolved] Generic Talend Job

_AnonymousUser — Sat, 01 Oct 2016 16:58:07 GMT

Hi All,
Can anybody tell me how to create dynamic schema using a text file where all the field names,datatype, length are present.
My requirement is
I have two files with different formats.For these two files i have to create a header and trailer for a set of records where header and trailer format is different for these two files. I have to create a generic module for header and trailer creation.
Hence i want to define the header structure of one file in one text file and the header structure of another file in another text file.
Using these two text files, during run time schema needs to be created in talend.
Is there any way? Please help me on this.

Re: [resolved] Generic Talend Job

_AnonymousUser — Wed, 22 Mar 2017 06:31:48 GMT

Hi
With community version, you have to create many subjobs for each file. With commercial subscription version, you are able to define a dynamic schema to map different structure, so that you only need one general Talend job for all files. For more information, please read this article:

Shong

Hi shong,
I got the exact same scenario. but i'm not being able to see the link you provided.
it's saying
" The page you were trying to reach could not be found. This could be because:

The page does not exist.
The page exists, but you do not have permission to view it."