Qlik Community

Qlik Connectors Discussions

Discussion Board for collaboration regarding Qlik Connectors.

Highlighted
a_leidenfrost
New Contributor II

QvRestConnector - problem with special characters / encoding

Hi all,

I use QvRestConnector to communicate with our REST API. 

CUSTOM CONNECT TO "Provider=QvRestConnector.exe;url=myApiMethod;timeout=30;method=POST;autoDetectResponseType=false;keyGenerationStrategy=-1;useWindowsAuthentication=false;useCertificate=No;certificateStoreLocation=CurrentUser;certificateStoreName=My;PaginationType=None;XUserId=maskedUser;XPassword=maskedPassword;";

let vBody = '{ "TeileNummer":"999999-TT", "Beschreibung":"Maßstab" }';

let vBody = Replace(vBody, '"', chr(34)&chr(34));

SQL SELECT 
"KomponenteID",
"TeileNummer", 
"Beschreibung",
"Preis"
 FROM JSON (wrap on) "root"
With Connection(
HTTPHEADER "Content-Type" "application/json; charset=utf-8",
BODY "$(vBody)");

The values for the json in the body are dummies - in production the values for "Teilenummer" und "Beschreibung" are taken from data that was previously loaded from a QVD file.

This works in most cases. The problem comes when one of the values contains special characters like ä, ü, or ß. In the body of the request that hits my webserver, these characters are replaces by question marks ('?').  

This seems to be an encoding problem - the json in a HttpRequest is assumed to be utf-8, but the strings are maybe written to the request body in ANSI? This seems very strange - from the documentation I gather that QVD files are always in utf-8, and the script as such is also encoded as utf-8. I'm not sure where a different encoding comes into play here.

What can I do to make sure that special characters in a QvRestConnector request body are transmitted correctly?

Labels (2)
1 Solution

Accepted Solutions
MVP & Luminary
MVP & Luminary

Re: QvRestConnector - problem with special characters / encoding

Encode those special characters before putting them in the body. I use a qvd that contains a list of characters and their encoding. Then I use the mapsubstring function to encode the body:

URL_Encoding_Reference:
MAPPING LOAD
    Character,
    URL_Encoding
FROM
    [$(vDataSiloQVDPath)\url_encode\url_encode.qvd]
    (qvd)
    ;

let vBody = '{ "TeileNummer":"999999-TT", "Beschreibung":"Maßstab" }';
let vBody = MapSubString('URL_Encoding_Reference','$(vBody)');

 

 


talk is cheap, supply exceeds demand
4 Replies
MVP & Luminary
MVP & Luminary

Re: QvRestConnector - problem with special characters / encoding

Encode those special characters before putting them in the body. I use a qvd that contains a list of characters and their encoding. Then I use the mapsubstring function to encode the body:

URL_Encoding_Reference:
MAPPING LOAD
    Character,
    URL_Encoding
FROM
    [$(vDataSiloQVDPath)\url_encode\url_encode.qvd]
    (qvd)
    ;

let vBody = '{ "TeileNummer":"999999-TT", "Beschreibung":"Maßstab" }';
let vBody = MapSubString('URL_Encoding_Reference','$(vBody)');

 

 


talk is cheap, supply exceeds demand
a_leidenfrost
New Contributor II

Re: QvRestConnector - problem with special characters / encoding

Thanks for the suggestion, but no, that doesn't work.

I'm posting with Content-Type application/json. All urlenconding does is that now I have percent signs in my strings instead of question marks. 

I still don't get why this is a thing in the first place. If my load script is in utf-8, the qvd data is in utf-8 and I want to have utf-8 in the request body, shouldn't it just work, without having to do extra encoding? 

MVP & Luminary
MVP & Luminary

Re: QvRestConnector - problem with special characters / encoding

Sorry, no idea then. If you're sure the values in the qvd are utf-8 then that should indeed just work.


talk is cheap, supply exceeds demand
a_leidenfrost
New Contributor II

Re: QvRestConnector - problem with special characters / encoding

Since I couldn't find a solution for using json, I bit the bullet and changed my Content-Type to application/x-www-form-urlencoded so that I could use your approach otherwise. Of course that means that I had to change the structure of the body data to conform to the content type.

It's not an ideal solution, but at least it works, so thank you again for your help.

Community Browser