Load Log File Encoding

[#1]

I recently completed recoding our OCX-based document reload code so that now it uses the QMS API instead.   Things are working pretty good so far, however, one thing that fell out of it is we are post-processing the reload script logs in order to pick out information, such as record counts that we use in a report on the results of the loads.   After converting the reload to use the API, I now find that the log encoding format is now UNICODE when it was ANSI under the OCX.   The documents themselves haven't changed, so it would appear to be something about how the reload is being invoked.

I've looked through the API for some possible property that might lead me to how to set this in the API to produce the reload log in ANSI.  Since we're still using the OCX in some instances, having to deal with log files in different formats is a PITA...  Does anyone know if it's possible to set this somehow?    I did find a Log File Encoding property in User Preferences but that seems to be a completely different context, I'm running server-side here via the QMS API, so User Preferences don't seem to be relevant...

--

Keith Doyle

[rwunderlich]

I deal with the same problem in my Script Log Analyzer tool. You can download the tool and see how I handle it in the script (see tab "Initial Logdata Load"):

Qlikview Cookbook: Script Log Analyzer http://qlikviewcookbook.com/recipes/download-info/script-log-analyzer/

What I do is test the first character of the logfile with various encodings  to see if I can interpret it as a number. When I get a positive I use that encoding to read the file. Probably would break if someone used a MMM-DD-YYYY format, but haven't had a complaint (yet). Here's the relevant code that determines the encoding.

FOR EACH vEncoding IN 'utf8', 'unicode', 'codepage is 1252';

  TempCode:

  FIRST 1 // Log first row

  LOAD IsNum(left(@1:n,1)) as X // Test first char. Set to true if it's a number.

  FROM

  [$(_logpath)]

  (fix, $(vEncoding))

  ;

  EXIT FOR WHEN peek('X'); // If a number, assume we have the right encoding.

NEXT vEncoding;

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

[#3]

So does that mean it's not possible to select the log file format when reloading through the API like it is with the OCX?   I see that the OCX does have "User Preferences" where I presume the option could be changed there.   But in the API, I'm creating a document task and then triggering it via EDX-- there is no equivalent to any of the User Preferences in that context?

It did occur to me the file could be examined to figure out what format it's in-- though our post processor is in VB .NET not QlikView script, I'm sure we can figure that out if we have to do it that way.   The thing is, we also have field support people that look at these files for diagnosing problems, and having them in different formats out in the field can be confusing at times.   Our API reloader is actually in C# and is my responsibility, so I guess one option could be to read the log files after the loads and re-write them in ANSI to make them backwards compatible with the rest of our code...

And I guess another possible solution might be to have the OCX code flip it to Unicode so that at least they're all in the same format.

But our first preference would really be to flip it to ANSI in C# via the QMS API if that were at all possible...

--

Keith Doyle

[rwunderlich]

I don't know if there is an available preference option to set the encoding when reloading on the server. Perhaps someone else on the forum knows or you might try QT Support with the question. My situation and solution is different because I need to handle logs from all sorts of environments outside my control. Good luck.

-Rob

[#5]

Ok, thanks.   In our case, we have dozens of our own internal logs and the QlikView logs get included in them because they're all generated on the same schedule and we scan them for information on the entire app.  It's particularly inconvenient to have them in mixed formats because so far they've always been 8-bit.  In the short run I've added logic to convert the log to 8-bits when they're copied into our central log repository which takes care of it for now.  But I will look further into whether or not that's necessary or if the server can be configured to generate ANSI logs.