topic Escape and Text Enclosure for csv in Talend Studio

Escape and Text Enclosure for csv

Gunner49 — Sat, 16 Nov 2024 01:01:05 GMT

Hi All,

I'm currently facing an issue while reading an input csv file where the data is enclosed within double quotes.

Below is a scenario where the data is as follows :

id;name;desc

1;"Mark";"Test Data"

2;"David";"Test Data Ha.""

3;"Steven";"\"

Here , within the data there are double quotes and a backslash within the desc column and I'm unable to escape both and hence the data is being shifted.

Ex : Escape char -

"""

Text Enclosure -

"""

Escape char - "\\" Text Enclosure - """

Also, the input files are huge (30+ GB) so would like to avoid having to use script. Other escape characters were also tried , but with no success. ("\"")

Any suggestions?

Thanks

Re: Escape and Text Enclosure for csv

manodwhb — Wed, 02 Dec 2020 14:43:48 GMT

@Nauman Hussain , you do one thing after tfileinputdelimited use the tJavarow or tMap to replace " with empty ,so that you can avoid the extra " .

Thanks,

Manohar

Re: Escape and Text Enclosure for csv

Gunner49 — Wed, 02 Dec 2020 16:54:53 GMT

Hi @Manohar B ,

Thank you.

That's an option for the 2nd record , but the 3rd record with desc column "\" is being shifted.

I can remove the double quote, but is there a way to handle the 3rd record ?

Here, the escape character and text enclosure are used as """

Input :

Output :

Thanks

Nauman