topic Re: tFileInputDelimited not read all fields of a data row in Talend Studio

tFileInputDelimited not read all fields of a data row

Anonymous — Sat, 16 Nov 2024 09:21:31 GMT

Hi all,

I have a data row in CSV file, I create a simple job to print it onto the console, but the tFileDelimited does not read all fields of the row.

The last field of the row is a Date value but when print it on console, the last value has been printed be "0.99"., I don't know what field it belong to.

How to read all the fields ?

I have tFileDelimited like this:

I want to keep the data format so CSV option was not checked and nothing checked in advanced settings

To see the whole post, download it here

Re: tFileInputDelimited not read all fields of a data row

Anonymous — Tue, 29 Aug 2017 09:34:06 GMT

A couple of things to check....

1) Have you set up enough columns in your metadata? I estimate (from a quick comma per row count) that you have over 1000 columns. Have you tried to add some more to see what result you get?

2) Do you have any carriage returns and line feeds in the data that you are not aware of? Your component config uses \r\n to separate rows. If it finds these, it will just start the next row. Since you are not using the CSV options, if you have \r\n in quoted text, this will be considered a new row.

Re: tFileInputDelimited not read all fields of a data row

Anonymous — Tue, 29 Aug 2017 09:59:02 GMT

Thank you,
1, I'm sure that the schema is enough column because other data row is print OK.
2, My data not have \r\n in quoted text, as you can see at the sample data I post.
I think my data row is out of max length which tFileInputDelimited can read by row.
Does Talend set the max length of each data row that tFileInputDelimited can read ?

Re: tFileInputDelimited not read all fields of a data row

TRF — Tue, 29 Aug 2017 09:59:14 GMT

Check your schema definition

Re: tFileInputDelimited not read all fields of a data row

Anonymous — Tue, 29 Aug 2017 10:00:16 GMT

I'm sure that the schema is enough columns because other data rows is print OK.

Re: tFileInputDelimited not read all fields of a data row

TRF — Tue, 29 Aug 2017 10:04:19 GMT

Do you have values like this one in other records?

"""Lg,Mx""16"""

Re: tFileInputDelimited not read all fields of a data row

Anonymous — Tue, 29 Aug 2017 10:21:57 GMT

I don't think I was very clear, the \r\n I was talking about are invisible. That text is the Java representative of a carriage return and line feed or simply hitting the "ENTER" key

like that.

@TRF raises a good point about the values you are expecting. If you are getting some rows returned with the correct number of columns and some which do not return the correct number of columns (or expected last column), it will likely mean one of the following things....

1) You have a carriage return and line feed in your row (some free text with the "ENTER" key having been pressed)

2) You have an extra comma in your free text (most likely)

There are online tools that will count the number of commas are in your rows. Try testing a "good" row against a "bad" row