topic Re: tFileInputDelimited with malformed data in Talend Studio

tFileInputDelimited with malformed data

Anonymous — Sat, 16 Nov 2024 08:10:56 GMT

Hi,

I have a 100000 line CSV file (tFileInputDelimited), which I'm trying to process with Tmap although experiencing an issue with one line:

"Everest","Norgay, Tenzing "Sherpa";Hillary, Ed","Nepal","Khumbu"

It creates extra columns, so the data no longer matches the schema

We're using CSV options:
Field separator: ","
Escape Char: "\"" (have tried different options)
Text Enclosure: "\""

Cheers, M

Re: tFileInputDelimited with malformed data

manodwhb — Thu, 31 May 2018 07:52:36 GMT

@mmeckens,you have extra Field separator,which were using as a Field separator.

You nee to correct the line in input csv file by manually.

below line data is not in proper csv format.,you need to correct it.

"Everest","Norgay, Tenzing "Sherpa";Hillary, Ed","Nepal","Khumbu"

Re: tFileInputDelimited with malformed data

TRF — Thu, 31 May 2018 08:04:05 GMT

If this issue arrives "sometimes", you may prefer to correct it automatically instead of manually.
For that, use a tFileRow component to read the file line by line, then using a tMap or tJavaRow, you will be able to reformat the line (maybe using some regex) to generate a well formated new file.