NPrinting report performance with Section Access

[Benno]

Hi,

I am working on a Report that uses Section Access. I use a server connection with "server authentication" and "apply user section access for reports". I use an HTML report with 2 pictures (each for one sheet in the QV application). I use filters to rebuild some actions and selections that need to be applied to the data (in total 3 filters, containing 4 Field filters and one Variable filter).  The filters are applied to the report and also to single objects in it. In my task I add the HTML report directly to the mail (no attachment). The final mail has around 430 KB.

The mail should be send to a group of around 130 store managers, each with their respective data.

All my test runs with the group as targets and only me as the recipient have taken around 2 hours to complete. Thats around 1  minute per mail.

My question is: What can I do to improve the performance?

And also what is a good or great performance for a report that uses section access, from your experience?

What I already did to try to improve the performance:

• Removed all sheets, objects and "Unsupported QlikView Document items, System Configurations and other limitations" (that are already mentioned in the support article) from the QV application
• Remove all the filters and only use section access
• Reduced the data from around 16 Mio. lines to 250.000
• Extended the configuration of the NPrinting Server from 2 cores with 8 GB to 8 cores with 32 GB
• Split the report into 2 reports and start them at the same time.

Sadly, all the above mentioned attempts had no effect on the performance.

If I understand the documentation correctly, then each core should create one report at a time. So with 8 cores I should be able to create 8 reports simultaneously. This doesn't seem to be the case to me.

Facts

• Qlik NPrinting April 2020,
• Windows Server 2012 R2
• Intel Xeon CPU E5-2690 v4 @ 2.60GHz, 8 Cores, 32 GB RAM
• QlikView April 2019 SR1, 
• Windows Server 2012 R2
• QV-File size:
• 341 MB - 571 Objects - 11 Sheets
• 96 MB - 119 Objects - 3 Sheets (reduced DataSet)
• Data model with 23 fields

Thanks for your help.

Best Regards,

Benno

[Lech_Miszkiewicz]

Hi @Benno 

to start i can only say that you have done all right things to improve the performance of your report. Now there are couple extra things you can do more:

1. Minimise all UI objects in QlikView used for this task so QV.exe client can navigate through them quicker (just in case there is any lag linked to the fact that object is still rendering before filters are applied)

2. Make sure you have proper setting done on your QlikView document-> Document settings--> server. If those are wrong it will have huge impact on performance! This is documented currently in June 2020 release notes but setting and performance drop applies to older versions: 

Logging slower performance of QVP connections
Jira issue ID: OP-9225
If your QVP connections perform slower than local ones, an informative log is written.
The workaround is to disable the Enable Push from Server option in the Document Settings page of the
QlikView Server.

instructions here: https://support.qlik.com/articles/000098313

3. Performance of NPrinting is based on performance of your QlikView server. It would be good to observe how is QlikView server coping the load from all 8 cores when your NPrinting task runs. This needs to be investigated as it may have a major impact on report distribution time.

thanks

Lech

[Benno]

Hi @Lech_Miszkiewicz

thanks for your tipps. I tried them, but sadly there was no improvement. Up until now I worked in our test environment. I also created a test case in our production environment. The behaviour there is the same.

When looking at the performance on the NPrinting server or the QlikView server I couldn't see a significant rise in the workload.

I saw in the resource monitor that the NPrinting server has several Qv.exe processes open, but all but one are in the status 'cancelled'. Is this normal / correct behaviour (see attached screenshot)?

It also looks to me as if the NPrinting server is caching the last 8 reports, because in some cases the reports of the last 8 recipients will be send instantly after starting the task, but the rest is then very slow again, which makes the overall time very slow. 

Could you please also tell me what is a good processing time (for 1, 10 or 100 mails) when using Section Access with QlikView.

Thanks

Benno

[Lech_Miszkiewicz · Accepted solution]

Hi @Benno 

There is no benchmark for "good" processing time. There is too many variable factors which may affect it. At this stage if you still think that your report is running slow you may want to consider contacting Qlik support. I have nothing to add. I also don't know about "cancelled" qv.exe.

Make sure you meet all system requirements. Check if you are running valid OS, that versions of QlikView server and Desktop are the same and supported, that service accounts are meeting all requirements. 

@Ruggero_Piccoli - would you add anything to this conversation?

by the way:

• for testing you can create "dummy" app using Ctrl+Q+Q script in QlikView add to it section access and try to run test report generation for the same users you are using in your production report.  
• the other test would be to remove section access and try to mimic it by applying user filters. 

[Benno]

Hi @Lech_Miszkiewicz,

Thanks again.  I tried your suggestion with the "dummy" app. I just added the dimension for our branches and no facts. I reached a time of around 8 minutes for 134 mails, just using a listbox to show that the report is correct. The report was also HTML with one picture of the whole sheet in it.

In the next step I added all the tables, fields and variables, that are essential for the report. Now I am landing at around 20-25 minutes. It's not perfect but much better than before.

Another thing I am encountering now is a recurring access problem from the Nprinting Engine "ERROR: NPrinting Engine failed to open document". At first I had this with 3 specific users. I had around 150 named users and groups authorized. I changed it to 3 groups with only the actual recipients, but now when I get an error, it's always a random person, normally 1 or 2, every other task run.

Have you experienced a similar behaviour or do you haven an idea where this could come from?

[Ruggero_Piccoli]

Hi,

So about the performances and the number of cores refer to https://help.qlik.com/en-US/nprinting/June2020/Content/NPrinting/DeployingQVNprinting/Performance.ht...

The Scheduler splits each report into chunks and send each to an Engine. Than the Scheduler collects all chunks and ask to an Engine to merge them. So one Engine is not working to a single whole report at a time. 

You did all good things to do in similar cases:

- create a .qvw for reporting with only the data and object used by the reports

- it should be no more needed to save all objects minimized but it is ok to do it for precaution

- avoid levels, pages etc that are all filters and slow down the generation speed

So at this point you can:

- check the delivery speed of the SMTP server. If I correctly understood you did all tests via email delivery, so the SMTP could be the bottleneck. Try to save the report into folders and compare the two executions time. 

- try to change the connection to a local copy of the qvw and compare the execution time To understand if the QlikView Server is or not the bottleneck.

Best Regards,

Ruggero