topic Re: create exception table column from multiple files in Talend Studio

create exception table column from multiple files

Anonymous — Sat, 16 Nov 2024 08:12:30 GMT

i have 4 CSV file which are having common transcation id,

but in,

file 1:- 1,2,3,4,5 rows,

file 2:- 1,2,3,6 rows,

file 3:- 1,3,8,7 rows,

file 4:- 1,4,7,8,9 rows.

so, i want to create table column which can be shown exception messsage that which file data is missing?

Re: create exception table column from multiple files

TRF — Mon, 28 May 2018 10:39:01 GMT

Based on your sample data, can you show the expected result?

Re: create exception table column from multiple files

Anonymous — Mon, 28 May 2018 10:58:30 GMT

Thanks @TRF for reply,

i have 4 CSV file which are having common transaction id,

but in, domain of transaction id is 0-9

file 1:- transaction id 1,2,3,4,5

file 2:- transaction id 1,2,3,6

file 3:- transaction id 1,3,8,7

file 4:- transaction id 1,4,7,8,9

target table:-

file name missing id

file 1 6,7,8,9

file 2 4,5,7,8,9

file 3 2,4,5,6,9

file 4 2,3,5,6

Re: create exception table column from multiple files

TRF — Mon, 28 May 2018 11:04:11 GMT

You may use a tFixedFlowInput component to build a pseudo-table with row 1, 2, 3, ..., 9.

Then, use tFileList to iterate over the file list, tFileInputDelimited to get the file content, tMap to join with the pseudo-table using an inner join and catch the rejected rows to have the missing transaction Id for each row.

Re: create exception table column from multiple files

Anonymous — Mon, 28 May 2018 14:01:02 GMT

thanks sir it works, but in the place of 1-9 row ,if we have milions of rows then what is the process?

Re: create exception table column from multiple files

TRF — Mon, 28 May 2018 14:09:40 GMT

Not the same story!

You need to merge all your input files (tFileList-(iterate)->tFileInpputDelimited-->tHashOutput) then remove duplicates (tHashInput-->tUniqRow-->tHashOutput).

This will replace the pseudo-table associated to a tFixedFlowInput in my previous answer.

Then you can continue with the scenario I proposed before.