topic Re: Reload taking too much time in QlikView

Reload taking too much time

Anonymous — Fri, 20 Nov 2020 18:02:30 GMT

Hi ,

I hv scheduled task for reload in morning earlier it was taking hardly 5 mins to reload but now its taking 2 hours for reload what cud be the problem plz suggest.

i checked the log files but could not found exact problem

thnx in advance

Re: Reload taking too much time

maleksafa — Wed, 03 Sep 2014 13:07:17 GMT

can you tell us more about the reload architecture that you are using? is this the final reload of the qvw? are you reading from qvds?

Re: Reload taking too much time

ashfaq_haseeb — Wed, 03 Sep 2014 13:16:44 GMT

Hi,

Try by deselecting Session Recovery and preload.

Check if that helped.

Regards

ASHFAQ

Re: Reload taking too much time

shane_spencer — Wed, 03 Sep 2014 13:17:04 GMT

Are you loading more data? more qvd's? do you have a stand alone Publisher Server or is it shared? Are you running out of Memory / Paging? have you restarted QlikView Distribution Service? (if so does it improve reload duration?) Have you checked # of Pinned Objects (as per below)? The more info you give the better the quality of the replies.

There's A Known Bug with .NET Framework 4

It was identified that there is a known bug with .NET Framework 4 that could be causing performance issues:

Reported to SE as Bug #49702 - closed as 3rd party bug:

This issue is caused by a memory leak in Microsoft FileSystemWatcher in .NET Framework 4.

The symptom is that tasks take longer and longer tor run. A restart of the Qlikview Distribution Service will temporary solve the problem. This has to do with the number of something called "Pinned objects". This number is usually in the region 0 to maybe a couple of hundred. At some customers it has been seen growing to millions. The number keeps growing until the Distribution service is restarted. As the number increased, so does the task execution time - until QDS is restarted. This can also be seen as the Distribution service consume a lot of CPU even when no tasks are running.

Re: Reload taking too much time

shane_spencer — Wed, 03 Sep 2014 13:17:37 GMT

It can be fixed by applying an updated detailed here:

http://support.microsoft.com/kb/2600217

Re: Reload taking too much time

Anonymous — Wed, 03 Sep 2014 13:33:22 GMT

yes its final reload of qvw from server im reading data from sql server database

Re: Reload taking too much time

maleksafa — Wed, 03 Sep 2014 13:36:16 GMT

maybe there is a problem with the connection to the sql database, is it local on the same network?

on the other hand i highly recommend you to use at least 2-tier architecture to prepare the data, the first qvw to prepare the data from sql server database and store it into qvds locally on the server and the second qvw will load the data from the qvd and have all the charts in it.

Re: Reload taking too much time

Anonymous — Wed, 03 Sep 2014 13:40:42 GMT

i face problem only when it runs automatically in morning ,but when i tried manually and/or setting reload trigger just for testing i didnt face any issue.its strange!!!!

Re: Reload taking too much time

ashfaq_haseeb — Wed, 03 Sep 2014 13:42:00 GMT

Hi,

Did you try my suggestion?

Regards

ASHFAQ

Re: Reload taking too much time

maleksafa — Wed, 03 Sep 2014 13:45:27 GMT

it must be a connection issue.

Re: Reload taking too much time

Wed, 03 Sep 2014 14:44:23 GMT

Just in case, are other tasks running on the server at the same time as the reload? Maybe the server is running other intensive tasks at the same time as the reload during the morning. This could be a reason for it to take a lot longer. It could happen even if the tasks are not QlikView related.

Re: Reload taking too much time

Anonymous — Thu, 04 Sep 2014 05:17:58 GMT

where is preload option??

Will it affect other functionality of qv???

Re: Reload taking too much time

ashfaq_haseeb — Thu, 04 Sep 2014 06:06:57 GMT

Hi,

Go to QEMC --> Document's tab --> User Documents --> Select your Application --> To right side you will find server tab --> go to performance sub tab --> and make supe preload is unchecked.

Secondly

go to QEMC--> System --> Setup --> QlikView Server --> Document tab --> make sure allow session recovery is unchecked.

Let me know if that solved your problem. Otherwise we will diagonose in other direction.

Regards

ASHFAQ

Re: Reload taking too much time

Colin-Albert — Thu, 04 Sep 2014 06:11:47 GMT

This could be a locking issue. Are there other tasks accessing the data, possibly a backup or an update task.

Can you add a nolock hint to your SQL query?

Re: Reload taking too much time

Anonymous — Thu, 04 Sep 2014 10:41:20 GMT

i hv made changes as per ur advice lets see what happens in tomorrow morning's load

Re: Reload taking too much time

ashfaq_haseeb — Thu, 04 Sep 2014 10:44:23 GMT

All the best.

Regards

ASHFAQ

Re: Reload taking too much time

Anonymous — Fri, 05 Sep 2014 05:02:18 GMT

i did not work still reload taking too much time for 1 dashboard and one dashboard failed...plz suggest

Re: Reload taking too much time

Anonymous — Fri, 05 Sep 2014 05:42:21 GMT

today i found this error in logs

(2014-09-05 06:30:47) Error: The task "Reload and distribute USD(Local) dashboard" failed. Exception:

(2014-09-05 06:30:47) Error: QDSMain.Exceptions.DistributionFailedException: Distribute failed with errors to follow. ---> QDSMain.Exceptions.DistributeToResourcesFailedException: Distribute to resource failed. ---> QDSMain.Exceptions.DscOffLineException: Failed to AddNamedUsers

(2014-09-05 06:30:47) Error: at QDSMain.Support.DSCHelper.AddNamedUsers(ILogBucket logBucket, XmlDocument xml, List`1 namedUsers, FileSystemPluginHelper fileSystemPluginHelper)

(2014-09-05 06:30:47) Error: at QDSMain.Support.DSCHelper.GetFileSystemSecurity(ILogBucket logBucket, FileSystemPluginHelper fileSystemPluginHelper, List`1 recipients)

(2014-09-05 06:30:47) Error: at QDSMain.QlikViewServerDistributionResource.CreateSettingsBlock(ILogBucket i_LogBucket, FileSystemPluginHelper i_FileSystemPluginHelper, String i_SourceDocumentName, DateTime i_LastUpdate, DateTime i_NextUpdate, Boolean i_EventDriven, String i_Category, eDistributionFileType i_TotalDistributionFileTypeMask, FilePackageContainer i_FilePackageContainer)

(2014-09-05 06:30:47) Error: at QDSMain.QlikViewServerDistributionResource.Distribute(ILogBucket i_LogBucket, DistributionRequest i_DistributionRequest)

(2014-09-05 06:30:47) Error: at QDSMain.DistributeTask.DistributeStreamToResources(Int32 round, Distribution distribution, IEnumerable`1 distributionResources)

(2014-09-05 06:30:47) Error: --- End of inner exception stack trace ---

(2014-09-05 06:30:47) Error: at QDSMain.DistributeTask.DistributeStreamToResources(Int32 round, Distribution distribution, IEnumerable`1 distributionResources)

(2014-09-05 06:30:47) Error: at QDSMain.DistributeTask.DistributionExecution(String fileName, Document qvbDocument, DistributionRequest distributionRequest, TaskResult taskResult, String repeatVariableName, String currentRepeatVariableValue)

(2014-09-05 06:30:47) Error: at QDSMain.DistributeTask.Distribution(String fileName, DistributionRequest distributionRequest, TaskResult taskResult, String repeatVariableName, String currentRepeatVariableValue)

(2014-09-05 06:30:47) Error: at QDSMain.DistributeTask.Execute(TaskResult currentTaskResult)

(2014-09-05 06:30:47) Error: --- End of inner exception stack trace ---

(2014-09-05 06:30:47) Error: at QDSMain.DistributeTask.Execute(TaskResult currentTaskResult)

(2014-09-05 06:30:47) Error: at QDSMain.Task.AbstractTask.TaskExecution(ILogBucket logBucket, TaskResult taskResult)

(2014-09-05 06:30:47) Information: Task Execute Duration=00:00:47.4522139

(2014-09-05 06:30:47) Information: TaskResult.status=Finished

(2014-09-05 06:30:47) Information: Notifying all triggers of new state:FinishedWithErrors

(2014-09-05 06:30:47) Information: Notifying all triggers of new state:FinishedWithErrors - completed

(2014-09-05 06:30:47) Information: Saving Task Result

Re: Reload taking too much time

flipside — Fri, 05 Sep 2014 06:41:53 GMT

Have you got Document Logs enabled in the documents - these logs will tell you whereabouts in your load script it is having a problem. Try comparing a manually loaded document log with a scheduled document log (you will either need to set the logs to be timestamped or rename the first log created before creating the second log). The failed job you had may be as a result of the first task tying up resources.

EDIT: have you got any logs stored at C:\ProgramData\QlikTech\DistributionService\1\Log\20140905\ (the date folder may be numbered differently). Look for the scheduled task/time folder and there may be a DocumentLog in there for the scheduled task.

Re: Reload taking too much time

ashfaq_haseeb — Fri, 05 Sep 2014 15:57:31 GMT

HI,

Can you send document log of the same.

secondly do you have any antivirus software running.

Can you let me know what happens if you schedule the same task at different time?

While that task runs do some of its generating qvd's were read in some other application at the same time, If yes you need to schedule dependent tasks.

Regards

ASJFAQ