topic Re: File Code Issues in QlikView

File Code Issues

ecrobles18 — Sat, 16 Nov 2024 21:22:55 GMT

Hello All,

I am experiencing an issue trying to read a file from a third party into QV. The original file was able to be read in with (txt, codepage is 1252, embedded labels, delimiter is ',', msq); but at some point the file format was changed and can only be read with (txt, unicode, embedded labels, delimiter is ',', msq); . I am now trying to test some historical data and cannot read both type of files at the same time. I have always read it using a loop so to load both files using different Character Sets. Is there a way to solution this issue? Any help with this matter is much appreciated.

Re: File Code Issues

p_verkooijen — Tue, 19 Feb 2019 18:56:52 GMT

Hi @ecrobles18

Try the suggestion Rob posted in this thread.

https://community.qlik.com/t5/QlikView-Management/Load-Log-File-Encoding/m-p/884711?advanced=false&collapse_discussion=true&q=test%20codepage%20unicode&search_type=thread

I deal with the same problem in my Script Log Analyzer tool. You can download the tool and see how I handle it in the script (see tab "Initial Logdata Load"):

Qlikview Cookbook: Script Log Analyzer http://qlikviewcookbook.com/recipes/download-info/script-log-analyzer/

What I do is test the first character of the logfile with various encodings to see if I can interpret it as a number. When I get a positive I use that encoding to read the file. Probably would break if someone used a MMM-DD-YYYY format, but haven't had a complaint (yet). Here's the relevant code that determines the encoding.

FOR EACH vEncoding IN 'utf8', 'unicode', 'codepage is 1252';

  TempCode:

  FIRST 1 // Log first row

  LOAD IsNum(left(@1:n,1)) as X // Test first char. Set to true if it's a number.

  FROM

  [$(_logpath)]

  (fix, $(vEncoding))

  ;

  EXIT FOR WHEN peek('X'); // If a number, assume we have the right encoding.

NEXT vEncoding;

Re: File Code Issues

p_verkooijen — Tue, 19 Feb 2019 19:01:48 GMT

Hi @ecrobles18

see Rob Wunderlich's post in this thread

https://community.qlik.com/t5/QlikView-Management/Load-Log-File-Encoding/m-p/884711?collapse_discussion=true&q=unicode%20codepage%20log&search_type=thread

"What I do is test the first character of the logfile with various encodings to see if I can interpret it as a number. When I get a positive I use that encoding to read the file. Probably would break if someone used a MMM-DD-YYYY format, but haven't had a complaint (yet). Here's the relevant code that determines the encoding."

FOR EACH vEncoding IN 'utf8', 'unicode', 'codepage is 1252';

  TempCode:

  FIRST 1 // Log first row

  LOAD IsNum(left(@1:n,1)) as X // Test first char. Set to true if it's a number.

  FROM

  [$(_logpath)]

  (fix, $(vEncoding))

  ;

  EXIT FOR WHEN peek('X'); // If a number, assume we have the right encoding.

NEXT vEncoding;

Re: File Code Issues

rwunderlich — Tue, 19 Feb 2019 19:33:02 GMT

I handle multiple files with different encodings in the Script Log Analyzer tool. You can take a look at the script.

Qlikview Cookbook: Script Log Analyzer http://qlikviewcookbook.com/recipes/download-info/script-log-analyzer/

The script bit that determines the encoding looks like this:

FOR EACH vEncoding IN 'utf8', 'unicode', 'codepage is 1252';
TempCode:
FIRST 1 // Log first row
LOAD IsNum(left(@1:n,1)) as X // Test first char. Set to true if it's a number.
FROM
[$(_logpath)]
(fix, $(vEncoding))
WHERE RecNo()>1 // Skip rec 1, which causes problems if it has a BOM (Byte Order Marker)
;
EXIT FOR WHEN peek('X'); // If a number, assume we have the right encoding.
NEXT vEncoding;

DROP TABLE TempCode;

and then $(vEncoding) is used in the LOAD statement. You may have to adjust the "IsNum(" test for your data. I know I'm looking for a number in that position, you may have to look at something specific to your data. If it's something you can recognize, then you have the right codepage for that file.

-Rob
http://masterssummit.com
http://qlikviewcookbook.com
http://www.easyqlik.com