tMap with Dynamic schema

[Anonymous]

Hi, 

I am pretty new to the talend tool. I am attempting to use talend tMap component to map data from two sources. However I also require the schemas to be dynamic as this is to be done for numerous tables or files therefore we want to escape having to define schema for each and using that in a tMap. What I've come across is that the output for this job is different when running a job within talend studio and eclipse. It's pretty strange as to why this is the case since I exported the exact same code(job) and built it in eclipse IDE. If anybody has ever had to use dynamic schemas together with tMap, I'd like some pointers as to how this can be achieved or possibly point out what I could be doing incorrectly. 

[med1]

Hello,

Could you please elaborate your cases ? I used dynamic schema and to handle the this schema i used the tjavarow.

[brama]

Hi,
If I understand correctly, you want to load different schema to target. If you are using a licensed version there is a data type dynamic itself in tmap. You can make use of it.

Thanks,
Ramachandran B

[Anonymous]

We have a scenario where we need to compare data from two sources say a file and table in a database and we need to do a comparison for each column in a file to each column in a database. We do this to identify any differences that may exist between the two sources of data. We have managed to achieve this by using a tMap component to compare each column/field in a table to each column/field in a file. However it's a lot of work if you're to do this for many tables/files, having to go through mapping each column to column in a tMap and end up designing many jobs that do exactly the same thing. So we now exploring the use of dynamic type for the schemas. I attached some screen shots on my first post.

[med1]

Hello,

I think for your case it is better to use dynamic schema.

So first try to select data from table to tmp file csv and then compare two files. If you have MDM module you can use specific component which allow to compare two files and give the distance between the columns etc..

BR,

Med

[Anonymous]

Thanks for your assistance, it's much appreciated.

In a case of using Dynamic schema, can it work correctly for joining data using a tMap component? Because what lead me here in the first place was the problem I got using a dynamic type to map my data.

I don't know of the MDM module, from components this is all I got (please see attached screen shot). I will lookup some YouTube tutorials on it.

Regards,

MDM.PNG

[med1 · Accepted solution]

Hello,

So as i explained in my last post, you can insert the data from table into tmp file.

After that you can use tJavarow to parse the both objects ( Dynamic schema is like java object).

Here is the code to parse the dynamic File :

Dynamic columns = row3.dynamic;
 
for (int i = 0; i < columns.getColumnCount(); i++) {  
    DynamicMetadata columnMetadata = columns.getColumnMetadata(i);  
    System.out.println(columnMetadata.getName() + ": " +
columns.getColumnValue(i) + "; " + columnMetadata.getType());
}

 There is an example in Talend Documentation, which explain how to handle the Dynamic schema :

https://help.talend.com/reader/KxVIhxtXBBFymmkkWJ~O4Q/WX1guuvSnxZg5nBBQ26LBQ

Good luck

[Anonymous]

Thank you so much. Your code solution works for my case.

[Anonymous]

Hi,
We do have the same scenario of comparing dynamic schema files.Could you please share us the solution ?

[ravi999]

This worked for one of my usecase. Thanks!