
Anonymous
Not applicable
2012-01-02
07:07 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How to create a dynamic schema mapping using Talend by reading schema.
Hi All,
i have to develop one such job which will be use to load all kind of file based on my distention table schema . i will have a XML file which has all the details of Input Type , File Type, Delimiter Type, Number of Columns, Columns Validation ( Trim, data Type, remove unwanted char), out put, table schema, columns data type, constraints. etc.
So the thing is whenever i am load file it should pic all the details from config. file, and then apply validation to convert or correct the data types with other validation to remove or format the data. then again read the destination table schema and map with source schema and then load to destination. this is sort of information. so please any one can tell how to and what to use to accomplish this task.
i have to develop one such job which will be use to load all kind of file based on my distention table schema . i will have a XML file which has all the details of Input Type , File Type, Delimiter Type, Number of Columns, Columns Validation ( Trim, data Type, remove unwanted char), out put, table schema, columns data type, constraints. etc.
So the thing is whenever i am load file it should pic all the details from config. file, and then apply validation to convert or correct the data types with other validation to remove or format the data. then again read the destination table schema and map with source schema and then load to destination. this is sort of information. so please any one can tell how to and what to use to accomplish this task.
981 Views
20 Replies

Anonymous
Not applicable
2014-05-28
07:53 AM
Author
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey,
Is it possible to accomplish the above task in Talend now?
Hey avdbrink,
Can you please guide how you accomplished the above task.
Thanks!
Is it possible to accomplish the above task in Talend now?
Hey avdbrink,
Can you please guide how you accomplished the above task.
Thanks!
306 Views

Anonymous
Not applicable
2014-05-28
09:41 AM
Author
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi ksultania,
this post has beeb published a long time ago ...
Now you've got dynamic schema but only with 'TIS' version.
regards
laurent
this post has beeb published a long time ago ...
Now you've got dynamic schema but only with 'TIS' version.
regards
laurent
306 Views

Anonymous
Not applicable
2014-05-28
09:54 AM
Author
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey kzone,
I came across Dynamic schema feature, but it helps only in taking dynamic schema from input source. Is it possible to map this schema to a fixed output schema dynamically?
Also, can you suggest where to learn from and how to learn talend to be productive?
I came across Dynamic schema feature, but it helps only in taking dynamic schema from input source. Is it possible to map this schema to a fixed output schema dynamically?
Also, can you suggest where to learn from and how to learn talend to be productive?
306 Views

Anonymous
Not applicable
2014-05-28
10:18 AM
Author
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
talend help is a great place to learn a lot about Talend.
for example with Dynamic schema : https://help.talend.com/pages/viewpage.action?pageId=5671283
regards
laurent
for example with Dynamic schema : https://help.talend.com/pages/viewpage.action?pageId=5671283
regards
laurent
306 Views

Anonymous
Not applicable
2014-05-29
01:18 AM
Author
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I came across Dynamic schema feature, but it helps only in taking dynamic schema from input source. Is it possible to map this schema to a fixed output schema dynamically? (that is making tMap feature dynamic)
Thanks for your suggestion.
Thanks for your suggestion.
306 Views

Anonymous
Not applicable
2014-05-29
03:43 AM
Author
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi ksultania,
For your scenario, it need to change column value dynamically.
We are working on your anther topic https://community.talend.com/t5/Archive/Mapping-Dynamic-Schema-to-a-Fixed-Destination-Schema/td-p/19..., then come back to you asap.
Best regards
Sabrina
For your scenario, it need to change column value dynamically.
We are working on your anther topic https://community.talend.com/t5/Archive/Mapping-Dynamic-Schema-to-a-Fixed-Destination-Schema/td-p/19..., then come back to you asap.
Best regards
Sabrina
306 Views

Anonymous
Not applicable
2014-05-29
04:11 AM
Author
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you.
Appreciate the help.
Appreciate the help.
306 Views

Anonymous
Not applicable
2015-12-28
06:05 AM
Author
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Any updates on the "Mapping dynamic schema to a Fixed Destination Schema" . we have a similar requirement, we are using dynamic schema feature for reading input files , now need to convert dynamic schema to fixed destination schema so that we can do validation on the columns using tmap(we have validation like column1> column2, Column2 > sysdate + 7 etc.
We are using Talend BigData 6.0 Version ( Licensed Version)
Thanks,
Sanjay
Any updates on the "Mapping dynamic schema to a Fixed Destination Schema" . we have a similar requirement, we are using dynamic schema feature for reading input files , now need to convert dynamic schema to fixed destination schema so that we can do validation on the columns using tmap(we have validation like column1> column2, Column2 > sysdate + 7 etc.
We are using Talend BigData 6.0 Version ( Licensed Version)
Thanks,
Sanjay
306 Views

Anonymous
Not applicable
2015-12-28
09:28 PM
Author
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Sanjay,
Have you already checked the related forum: https://community.talend.com/t5/Design-and-Development/resolved-Generic-Talend-Job/td-p/92354
Best regards
Sabrina
Have you already checked the related forum: https://community.talend.com/t5/Design-and-Development/resolved-Generic-Talend-Job/td-p/92354
Best regards
Sabrina
306 Views

Anonymous
Not applicable
2015-12-29
06:09 AM
Author
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Sabrina,
Here is the more detailed requirement.
We are extracting data from multiple sources like SQL-Server, Teradata and performing some check on the extracted data, the SQL is created dynamically based on the some preconditions for extracting the data. Suppose the extracted data contains 3 columns say col1 date datatype , col2 date datatype, col3 integer etc. and query returned 10 rows, now I need to validate each column values, these validation rules are also dynamic. we have different validation rules. For eg. validation rule is something like col1 > col 2
tTeradatainput---->tMap--->tfileoutputdelimited
tTeradatainput schema is defined as dynamic i,e we have created one column with dynamic data type. In tMap is there a way to split the dynamic schema columns and do the validation.
Here Output file schema should also be dynamic in nature ( based on the number of columns and data types, is there a way to create output schema dynamically).
We are open to use any other component in place of tMap, if required.
How can we achieve this in Talend
Thanks,
Sanjay
Here is the more detailed requirement.
We are extracting data from multiple sources like SQL-Server, Teradata and performing some check on the extracted data, the SQL is created dynamically based on the some preconditions for extracting the data. Suppose the extracted data contains 3 columns say col1 date datatype , col2 date datatype, col3 integer etc. and query returned 10 rows, now I need to validate each column values, these validation rules are also dynamic. we have different validation rules. For eg. validation rule is something like col1 > col 2
tTeradatainput---->tMap--->tfileoutputdelimited
tTeradatainput schema is defined as dynamic i,e we have created one column with dynamic data type. In tMap is there a way to split the dynamic schema columns and do the validation.
Here Output file schema should also be dynamic in nature ( based on the number of columns and data types, is there a way to create output schema dynamically).
We are open to use any other component in place of tMap, if required.
How can we achieve this in Talend
Thanks,
Sanjay
306 Views
