QMC Distribution Issue -- QDSMain.Exceptions.DistributeToResourcesFailedException

[genevamoores]

We are facing a problem with distribution which tips over when the user count goes approve a certain number.  We have managed to distribute the document fine if we reduce the number of users (amended the load script to say FIRST XXX on the security file) but as soon as we bump it up to the desired number of users, we get this error in the QMC is below. (The file generates fine in the SourceDocs but never gets copied to the server UserDocs).

Any insight from anyone? Is there a string limitation on the distribution list from a named parameter in the document? Is there any server parameter we can turn on to get more detailed logging information to help troubleshoot?

(7/15/2019 1:11:27 AM) Error: The task "QlikviewApplication" failed. Exception: || QDSMain.Exceptions.TaskFailedException: Task execution failed with errors to follow. ---> QDSMain.Exceptions.DistributeToResourcesFailedException: Distribute to resource failed. ---> System.Exception: l_FileSystemPluginHelper exception ---> QDSMain.Exceptions.DscOffLineException: Failed to AddNamedUsers || at QDSMain.Support.DSCHelper.AddNamedUsers(ILogBucket logBucket, XmlDocument xml, List`1 namedUsers, FileSystemPluginHelper fileSystemPluginHelper) ||at QDSMain.Support.DSCHelper.GetFileSystemSecurity(ILogBucket logBucket, FileSystemPluginHelper fileSystemPluginHelper, List`1 recipients) || at QDSMain.QlikViewServerDistributionResource.CreateSettingsBlock(ILogBucket i_LogBucket, FileSystemPluginHelper i_FileSystemPluginHelper, String i_SourceDocumentName, DateTime i_LastUpdate, DateTime i_NextUpdate, Boolean i_EventDriven, String i_Category, eDistributionFileType i_TotalDistributionFileTypeMask, FilePackageContainer i_FilePackageContainer) || at QDSMain.QlikViewServerDistributionResource.Distribute(ILogBucket i_LogBucket, DistributionRequest i_DistributionRequest) || --- End of inner exception stack trace --- || at QDSMain.QlikViewServerDistributionResource.Distribute(ILogBucket i_LogBucket, DistributionRequest i_DistributionRequest) || at QDSMain.DistributeTask.DistributeStreamToResources(Int32 round, Distribution distribution, Dictionary`2 offeredDistributionList, String fullNtfsPathToFile, UnifiedHubSettings unifiedHubSettings) || --- End of inner exception stack trace --- || at QDSMain.DistributeTask.DistributeStreamToResources(Int32 round, Distribution distribution, Dictionary`2 offeredDistributionList, String fullNtfsPathToFile, UnifiedHubSettings unifiedHubSettings) || at QDSMain.DistributeTask.DistributionExecution(String fileName, Document qvbDocument, DistributionRequest distributionRequest, IExecutingTaskResult executingTaskResult, String repeatVariableName, String currentRepeatVariableValue) || at QDSMain.DistributeTask.Distribution(String fileName, DistributionRequest distributionRequest, IExecutingTaskResult executingTaskResult, String repeatVariableName, String currentRepeatVariableValue) || at QDSMain.DistributeTask.PerformExecute(IExecutingTaskResult executingTaskResult) || --- End of inner exception stack trace --- || at QDSMain.DistributeTask.PerformExecute(IExecutingTaskResult executingTaskResult) || at QDSMain.Task.AbstractTask.TaskExecution(CurrentExecutionArgs args)

[Chip_Matejowsky]

Hello @genevamoores,

I have seen issues in the past where task distributions failed with error "QDSMain.Exceptions.DscOffLineException: Failed to AddNamedUsers..." being due to the task set to distribute to a user/group which had been removed from Active Directory.  More specifically, if task is set to distribute to "Supervisor" user account which has been removed from Active Directory. Have a look at Qlik Support article A QlikView Publisher task configured to distribute to "All Authenticated Users" or "All Users" fails... to see if it helps with this issue.

[genevamoores]

Thanks for your response.

That was my first suspicion and so I double checked that the active directory was very fresh and it did not fix the problem.

Also, we are distributing based on a parameter in the file and I see that the list is successfully reduced in the load process -- e.g. in the log file, it says

>>Dynamic Distribution for field PARAMETER returned 2461 possible recipients
>>Searching using "SAMAccountName" using parameter "PARAMETER" returned 2406 results.

So I would say it was successfully filtering based on valid ldap results already (these log entries were from previous successful runs).

AND, if I do a FIRST 200000 LOAD MYQVD for this security profile data, then the distribution succeeds. But If load the full 200284 records, the distribution fails. 

I feel that there is a limit on the length of the distribution list that is being generated and it is hitting an overflow condition but I don't know how to resolve or definitively prove that is the issue.

[Chip_Matejowsky]

Ah, that is good information to know. Are you seeing any errors in the QlikView Directory Service Connector (DSC) logs when this QDS task fails? If so, please post the errors to this thread.

[Chip_Matejowsky · Accepted solution]

@genevamoores I had another thought on what else you can try. The MaxReceivedMessageSize entry can be added/edited to the .config files for the QV services as needed. So try the following steps.

1. Stop the QDS, QMS, DSC services
2. Open a text editor (I use Notepad++), navigate to %ProgramFiles%\QlikView\ManagementService and open QVManagementService.exe.config file for editing
3. Scroll down to the MaxReceivedMessageSize entry and edit the default value of 262144 to 26214400 and save the file
4. Repeat step 3 for the QDS service in %ProgramFiles%\QlikView\DistributionService\QVDistributionService.exe.config (if the MaxReceivedMessageSize entry isn't present in the .config file, go ahead and add it)
5. Repeat step 3 for the DSC service in %ProgramFiles%\QlikView\DirectoryServiceConnector\QVDirectoryServiceConnector.exe.config (if the MaxReceivedMessageSize entry isn't present in the .config file, go ahead and add it)
6. Restart the QDS, QMS, DSC services
7. Run the reload/distribution task 

Please let us know how this goes.

[Brett_Bleess · Accepted solution]

This is most likely related the QDS DSC cache settings I believe.  In the QDS exe.config file, you may need to tweak the following lines:

<!-- Timeout in seconds for calls to the DSC-->
<add key="DSCTimeoutSeconds" value="120"/>
<add key="DSCCacheSeconds" value="900"/>

That config file resides in C:\Program Files\QlikView\Distribution Service by default.

The Timeout setting is for the service making a connection to the DSC, I would think 2 minutes should likely be fine, the other setting is the one you may need to change and increase to a larger number, but something else just came to mind as well, but I am not sure if it is in play or not.  I am wondering if due to the size of the user list, we are running out of cache space, the thing of which I am unsure is whether this is using Desktop Heap space or something else.  If it is related to Desktop Heap, generally that will throw a COM exception in the task log.  

The other thing you could do is in the DSC resource, in QMC, for the Plugin you are using, AD I expect, click the pencil icon out on the right side of the LDAP setup and you will see the same two settings as above there, you could increase the cache timeout there as well.  The only downside to increasing this is if something changes in the LDAP, we may not pick it up until the cache expires and we have to go back to the LDAP to query it again...

Hopefully this is making some sense.

Regards,
Brett