topic Re: N Squiggle or Spanish: eñe - shows non printing in SQL Server in Talend Studio

N Squiggle or Spanish: eñe - shows non printing in SQL Server

Tim_Ouimet — Fri, 15 Nov 2024 21:18:06 GMT

Hello!

I'm trying to troubleshoot my bulk data load issue - where I'm receiving a CSV feed of employee information - and loading to MS SQL Server.

The last name "Bañares" is displayed in the DB as "Ba�ares".

The file is loaded from a tFileInputDelimted component - through to a tDBOutput by way of a tMap. The encoding on inputdelimted is UTF-8.

The table definition for the field in SQL is nvarchar(150). The database collation is set to : SQL_Latin1_General_CP1_CS_AS

From what I'm reading - this should correctly be displayed in the database - but it's not. Is this a SQL Server collation issue - or am I missing something in my Import Routine?

Re: N Squiggle or Spanish: eñe - shows non printing in SQL Server

jlolling — Fri, 27 Oct 2023 08:53:16 GMT

First of all: char set conversations only applies to text reading or printing.

You should first check if the char set settings are correct for reading your file. Simply printout the values with a tLogRow.

If this is correct, the rest will work because the charset within the job code internally is always UTF-16 (Java standard) and this charset will be used from all database drivers and converted automatically into the charset of the database or table.

Sometimes database drivers need a hint which target charset should be used.

If you read now the value with a SQL tool and see something strange it also could happen the charset settings of the tool is wrong.

My guess is, the problem will be caused by a wrong actual char set of your file. You expect UTF-8 but the file has a different char set.

Re: N Squiggle or Spanish: eñe - shows non printing in SQL Server

jlolling — Fri, 27 Oct 2023 08:56:05 GMT

Another possible reason could be your special char is not part of the table charset Latin1. Did you checked that?

Re: N Squiggle or Spanish: eñe - shows non printing in SQL Server

Tim_Ouimet — Fri, 27 Oct 2023 11:31:57 GMT

The step immediately prior was taking the results of the JSON feed & exporting to file - which was set to an ISO charset. I changed them to UTF-8 and it was solved. Thank you