nPrinting - PeriodicCleaner exception in Scheduler log file

[StephanB]

Hi,

I am struggling to get nPrinting to cleanup physical report files.  The Reports to Keep and Days To Keep setting doesn't seem to have any influence on hard disc space management.  I have applied the proposed fix given here Support Article 000067475  but that did nothing, old reports still not deleted from HDD.

I had a look in the Scheduler's log file and found 191938 PeriodicCleaner exceptions logged over the last tow days.  The error message is:

Qlik.NPrinting.Scheduler 20.39.3.0 Qlik.NPrinting.Scheduler.Service.PeriodicCleaner 20210716T081927.281+02:00 ERROR NPRINT01 0 0 0 0 0 0 0 0 ERROR message : ERROR: System.NullReferenceException: Object reference not set to an instance of an object.↓↓ at Qlik.NPrinting.Repo.Service.PeriodicCleanupService.<>c__DisplayClass12_0.<DeleteItems>b__0(ISession s, Guid id) in C:\Jws\release-november2020-FugUOj5x6\server\NPrinting\src\Repo\Service\PeriodicCleanupService.cs:line 139

I have no clue why this is happening, and definitely not how to solve this.  For now I have scheduled a daily batch cmd to delete old reports to prevent situation where we run out of C:\ disc space.  (Previously the responsible app had 1.8 million undeleted files in the apps \reports\published folder.)

Any suggestions?

[Lech_Miszkiewicz]

Hi, 

The Reports to Keep and Days To Keep setting applies only to reports saved to News Stand (not to Folder HD). 

You mentioned that the proposed fix did not delete reports from HDD... so just want to clarify that you are talking about News Stand reports only. 

If you refer to reports stored as files in some folder you defined in your destinations then NPrinting does not clean those up.

cheers

[Lech_Miszkiewicz]

More info on that here: https://help.qlik.com/en-US/nprinting/November2020/Content/NPrinting/DistributionSchedulesAutomation...

[StephanB]

Thanks for the quick reply,

You mentioned that the proposed fix did not delete reports from HDD... so just want to clarify that you are talking about News Stand reports only. 

No, I only referring to the cleaning up of the physical report files.  The NewsStand reports do adhere to the Reports To Keep, and Days To Keep settings, i.e. the NewsStand is cleaned correctly, no problem here.

If you refer to reports stored as files in some folder you defined in your destinations then NPrinting does not clean those up.

Nope, I am not referring to purposefully set-up Destination/Folder report files.  I am referring to nPrinting's own internally kept report files in the C:\ProgramData\NPrinting\apps\f8308df5bc9a4eb986a6812116308f4e\reports\published folder for instance.  How is this managed by nPrinting?

Maybe I misunderstood the Support article 000067475 's last paragraph.  I am under the impression the second setting is applicable to the deleting of the physical files, and not the NewsStand reports. 

[Lech_Miszkiewicz]

I see...

So these are actually News Stand reports... or rather their phisical location on NPrinting server hards drive. 

Now, the question is why do you think this location is not cleared properly? Are there any specific instances of reports you know should not be there? If so, It would be good to know if the task which generated them was deleted, or maybe users dont exist anymore in NPrinting users list or maybe whole app , report and task was already removed...

What I am trying to say is, that usually orphan files are results of system not being able to perform standard cleanup due to the abovementioned.

I am not expert in this area so i guess you will get more info from guys like @Frank_S & @Ruggero_Piccoli . I hope they can share more light on the error and what you are experiencing. 

Sorry I could not help more.

cheers

[StephanB]

Sorry I could not help more. No, on the contrary, thank you very much for even trying to!! It is much appreciated!!!

Now, the question is why do you think this location is not cleared properly?  

About 2 weeks ago we got an alert from our server maintenance software that our nPrinting server is running out of C:\ drive space.  Upon investigation I found one specific nPrinting app's Report\Published folder to be 64Gb big, with approx 1.8 million files in it.  The oldest files were from April 2021 (when this specific report was deployed).  I had a look at the publish task of this report and it was set to Days to Keep = 1 and Reports to Keep = 1, and in the NewsStand this is confirmed.  (Sidenote the report runs 3 times a day and is publish to about a 1000 users, and with about 20 filters applied to it, hence the shear amount of reports generated (yes this seems excessive, but I am not involved in the report dev side ))

What I am trying to say is, that usually orphan files are results of system not being able to perform standard cleanup due to the abovementioned.

I hear you on the orphan side! Definitely something to keep in mind.  But in this case both the reports and publish tasks are still there, along with the users (granted, some users might have left the company and been removed from our AD (we sync user via LDAP to our AD), but it would be the minimum of orphaned causes)

[Ruggero_Piccoli · Accepted solution]

Hi,

Qlik NPrinting stores in folders like C:\ProgramData\NPrinting\apps\f8308df5bc9a4eb986a6812116308f4e\reports\published only the files related to reports published on the NewsStand. To minimize the used space you can set, as mentioned, the days to keep and reports to keep parameters and wait until the periodic cleaning job runs. If you want to set that job to run often set the parameters listed in the article you linked.

Starting from June 2020 each user can manually delete some specific reports:

Also in this case you have to wait the running of the cleaning job. There is not the equivalent administrator feature to delete all published reports for all users or similar. So you have to tell the users to manually delete their published reports.

For each published report there are also records on the Repository database. If you delete the files manually you have to search and delete also the related database records. The official documented features execute that procedure completely. I suggest to avoid to create not documented procedures or, at least, ask before implementing them. The error message that you posted Qlik.NPrinting.Scheduler.Service.PeriodicCleaner.. is due to the usage of the wrong cancellation method.  

If you have issues with the disk space check also the folder %ProgramData%\NPrinting because when upgrading Qlik NPrinting will create backups as documented in https://help.qlik.com/en-US/nprinting/May2021/Content/NPrinting/DeployingQVNprinting/Qlik-NPrinting-.... So there could be huge files.

Best Regards,

Ruggero

[Frank_S · Accepted solution]

Hi @StephanB

https://community.qlik.com/t5/Knowledge-Base/How-to-clean-up-Orphaned-NPrinting-database-file-folder...

As a tip, you may be able to leverage additional drive saving following the steps in the link above.

These steps will clean up app folder for apps (and any associated reports, connections etc) which have been deleted from your NP server via the NP web console.

If you have a NPrinting TEST environment, I suggest testing those steps in that environment before doing so in production.

Note: these steps will not clean up folders that are associated with NP App which are still in use on the NP server.

Kind regards...

[StephanB]

Hi to all,

Frank, Ruggero thanks for all the help and information so far, it is appreciated!!!!

However, the Accepted Solution(s) might be useful in other situations, but it does  not solve my HDD(physical disc) related space problems!  I am still stuck.

My one problem (nPrinting not cleaning up after itself) is still very much an issue with an active and existing task/report (i.e. no orphaned reports or deleted tasks) not cleaning its ../apps/###/reports/published folder on the c:\ drive.  (It does clean NewsStand, but the internal physical report files are not maintained).  And yes, the user can select All to delete, but, since the NewsStand is cleaned according to Keep settings, the delete all does nothing for the physical left over internal report files.

And concerning my other problem of the Scheduler log file being spammed with the Qlik.NPrinting.Scheduler.Service.PeriodicCleaner error, yes we now know where it comes from (usage of the wrong cancellation method), but how do I solve this? Has this been marked for fix in future release?

Regards

[Ruggero_Piccoli]

Hi,

The error Qlik.NPrinting.Scheduler.Service.PeriodicCleaner will be fixed in one of the upcoming releases. Refer to the release notes of new releases. 

In the folder ../apps/XYZ/reports/published there are all the reports published on the NewsStand generated from the app XYZ. So can you ask to all users to delete them from the NewsStand, wait one day and check if they were deleted also from the folder? Can you chen when the files were generated to see if there are other publishing tasks that are regenerating them? Can you upgrade to latest May 2021 SR1 and check if that solves? To be sure it is not a problem solved on more recent versions. I was to able to find any opened issue related to this. 

If you ask to all users to delete the published reports generated from the XYZ app and you stops all the publishing tasks that generate them are you able to see files after one day? It is a very strange behaviour that they are not deleted.

Best Regards,

Ruggero