topic Re: [resolved] Dynamic column mapping in TIS in Talend Studio

[resolved] Dynamic column mapping in TIS

djugal — Tue, 19 Feb 2013 07:21:53 GMT

hi,
can we pick up a csv data irrespective of the mapping,
e.g. i have 3 cols coming frm csv: name age sex
i want csv to mysql
i can do it the first tym but what if the column order is not constant. it can come in any sequence other tym
like age sex name etc.
can it be done by dynamic schema ? if so how ?

Re: [resolved] Dynamic column mapping in TIS

Anonymous — Tue, 19 Feb 2013 08:49:56 GMT

Hi,
Dynamics schema can deal with your issue as your described
See the related reference https://community.talend.com/t5/Design-and-Development/How-to-process-changing-data-structure/ta-p/21702
However, dynamic schema is only available in Talend Enterprise subscription version.
Best regards
Sabrina

Re: [resolved] Dynamic column mapping in TIS

djugal — Tue, 19 Feb 2013 08:54:00 GMT

thks sabrina

Re: [resolved] Dynamic column mapping in TIS

Anonymous — Tue, 19 Feb 2013 09:19:28 GMT

Hi,
Feel free to post your issue on forum, you are welcome.
Best regards
Sabrina

Re: [resolved] Dynamic column mapping in TIS

djugal — Tue, 19 Feb 2013 10:26:44 GMT

1 more question dynamic columns can be mapped directly to the target using Pass-through mode can i have an example of this ?

Re: [resolved] Dynamic column mapping in TIS

Anonymous — Wed, 20 Feb 2013 03:50:07 GMT

Hi,
There is no more examples about dynamic schema scenario except for the example in article I have given to you.
But for your requirement, I have designed a demo job for you.
The work flow is: tFileInPutDelimited-->tMySqlOutPut and the source file is:
name;age;sex
sabrina;26;f
shong;30;m
john;28;m
See my screenshots for more details. Hope it will help you.
Best regards
Sabrina

Re: [resolved] Dynamic column mapping in TIS

djugal — Wed, 20 Feb 2013 06:09:08 GMT

k but i mean what is the cols fro the csv are not in order
like name sex age instead of name age sex
in that case is there a way in dynamic schema we can map the fields
thks & regards,
djugal

Re: [resolved] Dynamic column mapping in TIS

Anonymous — Wed, 20 Feb 2013 06:29:20 GMT

Hi,
Do you mean there are three columns name,age and sex which arranged randomly.
Dynamic schema don't care the order and it just map the source file into target file. If the source file order is name,age and sex, the target file will correspond to that. If the source file order is age, name and sex, the target file must be age, name and sex.
If I misunderstand your meaning, would you mind giving more description to me?
Best regards
Sabrina

Re: [resolved] Dynamic column mapping in TIS

djugal — Wed, 20 Feb 2013 06:40:20 GMT

ok so i guess dynamic schema is not the thing i a looking for i guess,
see my prob is
i have a input file coming every hour in the schema : name sex age
mysql table has schema: name sex age
i have to run the job every hour to append the data.
my question was suppose at a time i got the schema of the input file in irregular sequence say name age sex
how can my job handle this jumble without errors. can this be achieved via dynamic schema ?

Re: [resolved] Dynamic column mapping in TIS

djugal — Wed, 20 Feb 2013 07:08:03 GMT

i found this link of bekwam's blog correct for my scenario
http://bekwam.blogspot.in/2011/06/dynamic-schemas-in-talend-open-studio.html
thks & regards,
djugal

Re: [resolved] Dynamic column mapping in TIS

Anonymous — Wed, 20 Feb 2013 07:40:53 GMT

Hi djugal,
Dynamic schema works for your case even though the sequence of columns varies in the source file.
BTW, the link you provided cannot be opened. Would you mind pasted the details content into the forum. I will appreciate it very much. Thanks a lot.
Best regards
Sabrina

Re: [resolved] Dynamic column mapping in TIS

djugal — Thu, 21 Feb 2013 12:38:41 GMT

hi sabrina,
can u elaborate how would dynamic schema work in my case ?
regards,
djugal

Re: [resolved] Dynamic column mapping in TIS

djugal — Thu, 21 Feb 2013 12:40:01 GMT

'http://bekwam.blogspot.in/2011/06/dynamic-schemas-in-talend-open-studio.html'

Re: [resolved] Dynamic column mapping in TIS

Anonymous — Fri, 22 Feb 2013 03:17:28 GMT

Hi djugal,
Let's still use your example:
1: In a first hour, inputting " name;age;sex" into your target DB, and then the data will be inserted into a table.
2: Here comes the second hour, inputting "age;name;sex" into your target DB, and then the data will be inserted into the existed table created by the first hour.
Source file for a first hour:
name;age;sex
sabrina;26;f
shong;30;m
john;28;m
Source file for the second hour:
age;name;sex
45;york;m
23;dan;m
18;mary;f
The actual result is : see pic
When the table has been created, the structure in DB is fixed. But dynamic schema will arrange your input data into the corresponding structure ignoring the order.
Best regards
Sabrina

Re: [resolved] Dynamic column mapping in TIS

djugal — Fri, 22 Feb 2013 07:55:09 GMT

Hi,
i have the same job structure as your example except oracle output,
filedelim -----> oracloutput
i am getting this error while executing in the logrow component:
ORA-00947: not enough values

Re: [resolved] Dynamic column mapping in TIS

Anonymous — Fri, 22 Feb 2013 08:39:14 GMT

Hi,

ORA-00947: not enough values

How many columns in your existed table? I think this issue is produced by missing insert column. For example, there are 3 columns age,sex,name in your table, but your input is just age,sex.
You can search it on Mr.Goolge to get more info about this issue description.
Best regards
Sabrina

Re: [resolved] Dynamic column mapping in TIS

djugal — Fri, 22 Feb 2013 10:05:17 GMT

Worked like a charm...............
Thanks Sabrina

Re: [resolved] Dynamic column mapping in TIS

Anonymous — Mon, 25 Feb 2013 05:48:10 GMT

Hi,

Thanks for your feedback.
Best regards
Sabrina

Re: [resolved] Dynamic column mapping in TIS

Anonymous — Tue, 15 Jul 2014 12:37:28 GMT

Hey,
This post is little old, but i wanted to know something related to it.
What if the column names are also different? say name in one file and names in other. can we map it dynamically(after reading a file or something)?

Re: [resolved] Dynamic column mapping in TIS

Anonymous — Wed, 16 Jul 2014 04:45:17 GMT

Hi ksultania,
Here is a response in your another forum https://community.talend.com/t5/Design-and-Development/resolved-Dynamic-Column-Mapping/td-p/109075, could you please take a look at it?
Best regards
Sabrina