QlikSense - Load Script - Adding Large XML files

[ArnadoSandoval]

mto‌

We are experiencing serious issues with QlikSense when trying to add large XML files to its loader script; look at the picture below; with Qlik Sense the looping (busy) cursor (the circle in the middle) sits there, any attempt to do anything is ignored by Qlik Sense, the user frustration is high, and become higher when running the script (the application was originally written in QlikView) when the script freezes.

I just tried the same XML file with QlikView, and we were able to open, select/de-select tables and columns in less that 60 seconds !!! and we generate the QVDs yesterday using the original QlikView QVD Creator; it took less than 40 minutes.

Qlik Sense freezes!

The size of the XML file is 115,367 KB.

Michael, we need assistance with this issue, anything you can suggest will be appreciated; with QlikSense we already change the way the script loads data from these XML files.

[ArnadoSandoval]

This is an update:

G’Day All,

This is an update regarding this issue that could shed some light:

1. I create a brand new QlikSense application, selecting all the tables from the first XML file (QS did not allow to select a single table)
2. The script does not have a LOOP.
3. Its first run freeze at the offending table: RouteVariantPoint
4. We commented out everything before and after the section loading RouteVariantPoint, no dramas.
5. Then, we added an alias before the LOAD, like this:

RouteVariantPoint:

LOAD

…

The process froze!!!

• It seems the alias has something to do with the issue, but it breaks when the for/loop is re-introduced.

[rdraeger]

Arnaldo,

I have created a tiny test application and can confirm that the error occurs later if the table doesn't have an alias. It does, however, still occur.

This is my script:

set a = 1;

do while a < 5000


[Content]:
LOAD [Line]
 FROM [lib://Source/test.xml]
(XmlSimple, table is Content);

let a= a+1;

loop

and this my xml-file:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Content>
     <Line>Hello World</Line>
</Content>

whith the Alias, the script stops within the first 300 records, without within the first 3000 records.

I guess, we'll have to wait for a bug fix by Qlik.

Cheers,

Robert

[rwunderlich]

Hi Robert,

I ran your test script without problem in QS Desktop Sept 2017 release.  Did you test in desktop or server?

-Rob

[ArnadoSandoval]

Hi Robert,

I ran the test 31 times, it failed 3 times (I uploaded my test notes); I am using: Qlik Sense Desktop September 2017: 11.14.3; Its first run was a failure but I did not record the number of loops done, it failed again after adding a simple UI, 24 tests later it failed by the third time, this time I was running Outlook Rules and dragging XL to the second screen. The reason to be running Outlook Rules is because I was running them earlier, near the time I started these tests.

My laptop is running Windows 8, with 16 GB RAM, Intel i7 @ 2.50 GHz

[ArnadoSandoval]

Keeping everyone up to date.

Tests Done Yesterday:

• Our local Qlik Supplier suggested to disable the ProhibitDTD parameter in the Settings.ini file, so we added the line:

        ProhibitDTD=0;

• The QlikSense Application loading the tables from just the first XML is now loading the first 12 tables before the one causing the freeze, which is also loaded, the approach has been re-introducing a table to the script, running the script between 5 to 10 times; sometimes the process hang, forcing us to close QlikSense, if the process keep hanging at a particular place we play with the alias until all work fine, and we keep going with the next table. It peculiar that once a table loads, it no longer freeze the process.

[rwunderlich]

I ran it a few more times and got it to hang intermittently.  Can't see what the pattern may be yet.

-Rob

[rdraeger]

I can't see a pattern, either. Originally, we encountered the issue using the Qlik Deployment Framework (QDF), which relies heavily on reading QVD-metadata. On some development machines the process stopped every other run and on others the issue only occured a few times.

At first I thought, that it was some type of deadlock, because of something like that:

[$(vL.QDF.QVTableName)]:
LOAD 
    ...
    QvdTableName('$(vL.QDF.filelist)')   as QVTableName,
    ...
    [CreatorDoc] as QVTableCreator
    FROM [$(vL.QDF.filelist)] (XmlSimple, Table is [QvdTableHeader]);     

because QvdTableName also reads the file that the load statement loads. Therefore I refactored those statements but the error still occured.

The support told me that there is a new XML Parser being developed for the next release and hopefully the issue will be solved then.

Cheers,

Robert