tFileInputDelimited with malformed data

[Anonymous]

Hi,

I have a 100000 line CSV file (tFileInputDelimited), which I'm trying to process with Tmap although experiencing an issue with one line:

"Everest","Norgay, Tenzing "Sherpa";Hillary, Ed","Nepal","Khumbu"

It creates extra columns, so the data no longer matches the schema

We're using CSV options:
Field separator: ","
Escape Char: "\"" (have tried different options)
Text Enclosure: "\""

Cheers, M

[manodwhb]

@mmeckens,you have extra Field separator,which were using as a Field separator.

You nee to correct the line in input csv file by manually.

below line data is not in proper csv format.,you need to correct it.

"Everest","Norgay, Tenzing "Sherpa";Hillary, Ed","Nepal","Khumbu"

[TRF]

If this issue arrives "sometimes", you may prefer to correct it automatically instead of manually.
For that, use a tFileRow component to read the file line by line, then using a tMap or tJavaRow, you will be able to reformat the line (maybe using some regex) to generate a well formated new file.