topic Re: [resolved] Get input file row by row as a string in Talend Studio

[resolved] Get input file row by row as a string

Anonymous — Sat, 16 Nov 2024 13:41:17 GMT

Hi, I apologize if this have been answered before.
TOS 3.2.0, Java
I have simple job: Get the list of files from a directory and parse one by one and store data in database table:
tFileList => tFileInputDelimited => tMap => Database Table
Each file row has more than 100 ~ delimited values, and each file has sometimes 10000 rows.
I use tMap to extract few of values in each row and map to a field in database table.
One of the values is file name itself which I got like this: ((String)globalMap.get("tFileList_1_CURRENT_FILE"))
One of the field in the table is current file row itself. Since I have more that a 100 delimited columns it would be a pain to concatenate each individual column. Is there a variable that I can use to reference entire row (as a string) as it comes from the file.
Thanks in advance.

Re: [resolved] Get input file row by row as a string

Anonymous — Fri, 13 Nov 2009 03:12:24 GMT

Hello

One of the field in the table is current file row itself. Since I have more that a 100 delimited columns it would be a pain to concatenate each individual column.

You can use tFileInputFullRow component to read the whole line as one row.
Best regards

shong

Re: [resolved] Get input file row by row as a string

Anonymous — Mon, 16 Nov 2009 16:05:17 GMT

Thanks Shong, but I also need to extract few elements from that line.
tFileInputFullRow let me extract just whole row and tfileInputDelimited let me extract delimited values that I need.
What I need to map is 3 elements from file delimited row and a whole row itself. I there a way to do that. is there variable in tfileInputDelimited that holds whole row string.
Thanks.

Re: [resolved] Get input file row by row as a string

Anonymous — Tue, 17 Nov 2009 06:10:13 GMT

Hello
tFileInputFullRow ---row1-->tFlowToIterate--iterate-->tFixedFlowInput--->tExtractDlimitedFields--tMap---tMysqlOutput
On tFixedFlowInput, only generate one row with one column, set the value of column as: (String)globalMap.get("row1.line")
On tExtractDlimitedFields, extract the delimited values
On tmap, add a new column called line, set the expression as: (String)globalMap.get("row1.line")
if you need to filter some columns, doing that on another tmap or tFilterColumns component.
about tFlowToIterate, see 5827.
Best regards

shong

Re: [resolved] Get input file row by row as a string

Anonymous — Tue, 01 Dec 2009 18:10:42 GMT

perfect, thanks....

Re: [resolved] Get input file row by row as a string

_AnonymousUser — Thu, 25 Aug 2016 07:11:20 GMT

Dear Shong,
tFileInputFullRow ---row1-->tFlowToIterate--iterate-->tFixedFlowInput--->tExtractDlimitedFields--tMap---tMysqlOutput in this flow if I have to extract data for fixed width file for each row with different length, which component i should use in place of tExtractDlimitedFields?

Re: [resolved] Get input file row by row as a string

Anonymous — Thu, 25 Aug 2016 10:17:26 GMT

Dear Shong,
tFileInputFullRow ---row1-->tFlowToIterate--iterate-->tFixedFlowInput--->tExtractDlimitedFields--tMap---tMysqlOutput in this flow if I have to extract data for fixed width file for each row with different length, which component i should use in place of tExtractDlimitedFields?

Take a look at tExtractPositionalFields component.
Regards
Shong