How to remove duplicates from final output

[AV2022]

Hello Team,

I have connected two Input components to extract data from 2 different tables from my source DB which is the Access DB. Exporting the data from both the tables are working fine and i am able to import into my Oracle tables successfully. Attached screenshot shows how i have connected the components.

My issue is i have removed the duplicate entries using tUniqrow for both the input tables before it is connected to tMap, when the data is moved from tMap to respective output oracle tables i am still seeing duplicate entries, basically the data from 2 different input tables are pushed to same output oracle tables

For Example: Input Table Values InputTable2 Values

1 1

2 3

Output Oracle table Values

1

1

2

3

Required result : My output table should not have two values of 1

I am not sure if its the problem with the way i have connected the tUniqrow component or not. Kindly help me with this issue.

[Anonymous]

Cache the two outputs into memory using tHashOutput component. In Next subjob, read the data back from memory using tHashInput, and link it to tUniqrow to remove the duplicated records, write the result to Oracle table.

Regards

Shong

[AV2022]

Hello Shong,

Thanks for the response, but unfortunately i am not able to find the tHash component in the palette. Please see the screenshot, kindly suggest us what needs to be done.

 

Thanks

Anand V

[AV2022]

Hello Shong,

Please ignore the above comment. tHash components where hidden in the panel. I have added it to the palette. Let me try the solution which you have suggested and update you back

Thanks

ANand V

[AV2022]

Hello Shong,

I am bit confused here, can u share a screenshot which shows how i need to connect the tHash components, in my case,do i need to connect the tHashOutput component after tMap? please clarify

Thanks

Anand V

[jlolling]

Could you show us the component config of the tUniqueRow. Actually your design should work well.

[AV2022]

Hello,

Please find the component config details as requested. When i extract the data from single input table by using tUniqueRow iam getting the desired output, problem occurs when data is extracted from multiple tables and the input from both tables are pushed to same oracle output tables. The values of N_PROCESSO should be duplicate in the output tables

Note: Both the Input tables have the same column N_PROCESSO