topic Re: Cycles vs Multiple Reports with Filters - vast difference in processing speed in Qlik NPrinting

Cycles vs Multiple Reports with Filters - vast difference in processing speed

IAS_Eddie — Tue, 16 Apr 2019 16:37:21 GMT

Good morning,

I have an excel NPrinting report with data that is 82 columns wide and 100,000+ lines long total. I need separate files for each product. My first thought/test was to cycle by the product field, but the NPrinting report takes 20-30 minutes to create the excel files with just the raw data and some simple set analysis (mentioned below). A right click export on the table within Qlik is done in about 60 seconds.

However, if I create different reports, each with individual product filters, NPrinting can create all the formatted excel files in less than five minutes.

Is this typical NPrinting behavior? Is there any sort of optimization that can be done for large data sets?

I opted for speed through separate NPrinting reports when they were simple, but now I have to go back and add multiple pivot tables to each report, which has made me rethink cycling.

There are no difficult calculations within the data. It's mostly just a raw data dump separating the data into two buckets with set analysis expressions plus half a dozen pivot tables I'll be adding.

Set Analysis expression used:

TransactionCode -= {'XC'}

and

TransactionCode = {'XC'}

I appreciate any insight. Thanks!

Re: Cycles vs Multiple Reports with Filters - vast difference in processing speed

Lech_Miszkiewicz — Tue, 16 Apr 2019 23:42:40 GMT

Cycle or Filter functionality is still a Qlik selection (Filter) which:

needs to be applied on your Qlik Application,
whole data model needs to recalculate and...
...return readiness of the object (table with your data) to be exported (evaluate if there is data for applied filter)
export data to your xls file (temporary)
move on to next value in your cycle
repeat above steps for each value in cycle

At the end all files are transported from temp to actual destinations (distribution step)

The more values you have in field you cycle through the more it is going to take to produce report.

it is not strange to me that exporting whole data set takes only few seconds, maybe a minute... The part which takes time is the whole process of selecting recalculating set etc... which i described above.

How long does it take directly in Qlik to calculate your super wide table each time you apply selection on different product? If you can optimise calc time on this table (for example by implementing calculation condition like: Only(Product) or similar...) it will definitely help reduce report production time. If it is already instant then there is not much you can do - sorry!

BTW. Can you specify version of Qlik and NPrinting you are using? It is important that you use the latest NPrinting to ensure the best possible performance...

thanks

Lech

Re: Cycles vs Multiple Reports with Filters - vast difference in processing speed

IAS_Eddie — Wed, 17 Apr 2019 13:25:01 GMT

I appreciate the insights on how the cycling works.

The product selection within Qlik is instant and we're using NPrinting Nov 2018: 18.39.6.0 and Qlik Sense Nov 2018 Patch 2.

Thanks

Re: Cycles vs Multiple Reports with Filters - vast difference in processing speed

IAS_Eddie — Wed, 17 Apr 2019 14:30:23 GMT

So I have a bit of an update. I ended up trying to run the report again, but this time without a cycle. All the products were lumped together with half a dozen pivot charts based on the data set.

Even without the cycle, it still took 25 minutes for the report to load and be ready for download, 60 MB file.

I understand that NPrinting probably wasn't designed with a report like this in mind, but it makes me think that when a single report is ran, the system isn't taking full advantage (parallel processing?) of it's resources as opposed to when I split it up into multiple reports and run them simultaneously. I'm no IT expert, so I might just be making stuff up!

Thanks