topic Re: tFileInputDelimited reads fewer rows than lines in the file in Talend Studio

tFileInputDelimited reads fewer rows than lines in the file

mickg — Sat, 16 Nov 2024 04:45:42 GMT

I have a UTF-8 csv file with 248,995 LF-separated lines (header included).

tFileInputDelimited reads only 248,980 data rows, no rejects.

What should I be looking for in the data? Or in the component?

Re: tFileInputDelimited reads fewer rows than lines in the file

TRF — Thu, 05 Sep 2019 23:06:01 GMT

Maybe some lines with 1 or more LF in the middle.

Re: tFileInputDelimited reads fewer rows than lines in the file

mickg — Fri, 06 Sep 2019 21:05:14 GMT

Thank you, TRF.

Excel and Talend (if CSV specified) are able to keep together in one field anything between two "s, including control characters.

So even if the number of lines of a .csv file (as read in notepad++) is higher than the number of rows read with Excel are different (adjusted for headers), the data rows are correct.

Not sure what the end system does with the LF's in a field...

Re: tFileInputDelimited reads fewer rows than lines in the file

TRF — Sat, 07 Sep 2019 14:00:11 GMT

If your fields are contained between "s and you use Notepad++ to count lines, it's normal to have a difference with tFileInputDelimited as this one counts logical records, not physical lines.
If you count lines with tFileRowCount or read files with tFileInputFullRow you will have the same count as with Notepad++.