Qlik Community

Qlik Sense Data Connectivity

Discussion board where members can learn more about Qlik Sense Data Connectivity.

flaurntiu
New Contributor III

Loading data with Polish characters from QVD these are gone

Currently we have a customer that uses Polish characters in their SQL database. These are shown in the result set when running SQL Select statements.

When using the same select directly in Qlik Sense, the characters are shown in Qlik Sense apps.

When the data is loaded from SQL in a Qlik sense App and then written to a QVD file, the Polish characters seem to be gone. Retrieving the QVD contents using Q-Eye will not show these characters. Loading data from the QVD into a Qlik Sense app does not show the characters.

I have seen this remotely and am waiting on the customer SQL database to be uploaded for further testing.

Does anyone know how to fix this?

Tags (1)
1 Solution

Accepted Solutions
flaurntiu
New Contributor III

Re: Loading data with Polish characters from QVD these are gone

I seem to have found the cause for this.
When loading the data from the SQL server, it can be in several code pages. When the SQL field contains special characters it will load them properly when read from the database. However if the data is not in Unicode format (e.g. nvarchar, nchar, etc.)  the data will not be stored properly into a QVD file. The special characters will be substituted by a non-Unicode counterpart.This means that the character 'Ł' will be replaced by 'L', 'ą' by 'a' and so on.

Therefore you will need to rewrite the SQL queries to cast or convert the text fields to Unicode format.

An example:

load accounts sample.png

This results in the following QVD contents (part of) for the Accounts1 table without Unicode:

Accounts1_qvd file.png

This results in the following QVD contents (part of) for the Accounts2 table with Unicode:

Accounts2_qvd file.png

As you can see the second file keeps the special characters, as they are recognized as unicode characters and stored properly in the QVD.
Hope this helps others running into this issue as well.
Have fun Qlik-ing around

2 Replies
flaurntiu
New Contributor III

Re: Loading data with Polish characters from QVD these are gone

Could this have something to do with issue QLIK-67044?
The customer is now using Qlik Sense 3.1.x

flaurntiu
New Contributor III

Re: Loading data with Polish characters from QVD these are gone

I seem to have found the cause for this.
When loading the data from the SQL server, it can be in several code pages. When the SQL field contains special characters it will load them properly when read from the database. However if the data is not in Unicode format (e.g. nvarchar, nchar, etc.)  the data will not be stored properly into a QVD file. The special characters will be substituted by a non-Unicode counterpart.This means that the character 'Ł' will be replaced by 'L', 'ą' by 'a' and so on.

Therefore you will need to rewrite the SQL queries to cast or convert the text fields to Unicode format.

An example:

load accounts sample.png

This results in the following QVD contents (part of) for the Accounts1 table without Unicode:

Accounts1_qvd file.png

This results in the following QVD contents (part of) for the Accounts2 table with Unicode:

Accounts2_qvd file.png

As you can see the second file keeps the special characters, as they are recognized as unicode characters and stored properly in the QVD.
Hope this helps others running into this issue as well.
Have fun Qlik-ing around

Community Browser