16 Replies Latest reply: Apr 10, 2017 8:50 AM by Carola Kofler RSS

    Reload with Section Access under different user fails

    Carola Kofler

      I am working on a project, where access to the qvw should be extremely limited. Due to security reasons, the general service user which runs all services on the Qlikview Server should not have access to this specific qvw file.

      I was attempting to work with the task option "Reload" - "Section Access" - Username/Password - but all attempts to reload the document via the task fail. If I reload the document directly from the client when connected as the same user, all goes well.

      Below the error log from the reload.

       

      We are running Qlikview 12 SR5.

       

      (2017-04-06 09:09:22) Information: Starting task 'xxxxxxx'. Id:7e0fda33-8d13-4efa-a8ab-03d8e2d7722a. Triggered by 'ManualStartTrigger'. Id:00000001-0002-0003-0405-0607080a0b0c

      (2017-04-06 09:09:22) Information: Entering Task Execution.

      (2017-04-06 09:09:22) Information: ClusterID=1

      (2017-04-06 09:09:22) Information: QDSID=3030ed33-4bf2-84dc-c7c1-2af29e157ffe

      (2017-04-06 09:09:22) Information: TaskID=7e0fda33-8d13-4efa-a8ab-03d8e2d7722a

      (2017-04-06 09:09:22) Information: MaxRunTime=1.00:00:00

      (2017-04-06 09:09:22) Information: MachineName=QLIKVIEW1

      (2017-04-06 09:09:22) Information: Max attempts:1

      (2017-04-06 09:09:22) Information: Current Attempt=0

      (2017-04-06 09:09:22) Information: Task Dependencies are OK

      (2017-04-06 09:09:22) Information: Document is marked to be Reloaded with fresh data. Initializing Reload for Distribution.

      (2017-04-06 09:09:22) Information: Opening "C:\QlikView_SourceDocs\xxxxxxxxx.qvw"

      (2017-04-06 09:09:22) Information: Allocating new QlikView Engine. Current usage count=2 of 4 (of type non-reader).

      (2017-04-06 09:09:22) Information: Max retries:5

      (2017-04-06 09:09:22) Information: Attempt:01

      (2017-04-06 09:09:24) Information: Opened the QlikView Engine successfully. ProcessID=14132

      (2017-04-06 09:09:24) Information: Allocated QlikView Engine successfully. Current usage count=3 of 4 (of type non-reader). Ticket number=4194.

      (2017-04-06 09:09:24) Information: Loading document "C:\QlikView_SourceDocs\xxxxxx.qvw" (0.20 Mb)

      (2017-04-06 09:09:25) Information: Physical FileSize=0.20 Mb. Memory Allocation Delta for this file=4.73 Mb. Available Physical Memory Before Open=599.11 Mb. Available Physical Memory After Open=586.11 Mb. Total Physical Memory=32767.55 Mb.

      (2017-04-06 09:09:25) Information: Attempted to load the document without data.

      (2017-04-06 09:09:25) Information: The document was loaded successfully.

      (2017-04-06 09:09:25) Information: Document was opened successfully

      (2017-04-06 09:09:25) Information: Starting reload

      (2017-04-06 09:09:25) Information: The Source Document is being reloaded. DocumentPath=C:\QlikView_SourceDocs\xxxxxxx.qvw

      (2017-04-06 09:09:25) Information: The Source Document reload complete. DocumentPath=C:\QlikView_SourceDocs\xxxxxxx.qvw

      (2017-04-06 09:09:25) Information: Memory Allocation Delta for this file=4.82 Mb. Available Physical Memory Before Reload=584.70 Mb. Available Physical Memory After Reload=586.32 Mb. Total Physical Memory=586.32 Mb.

      (2017-04-06 09:09:25) Error: The Source Document was NOT reloaded successfully. DocumentPath=C:\QlikView_SourceDocs\xxxxxxx.qvw.

      (2017-04-06 09:09:25) Information: Closing the document.

      (2017-04-06 09:09:26) Information: Closed the QlikView Engine successfully. ProcessID=14132

      (2017-04-06 09:09:26) Error: The task "xxxxxxxx" failed. Exception: || QDSMain.Exceptions.TaskFailedException: Task execution failed with errors to follow. ---> QDSMain.Exceptions.ReloadFailedException: Reload failed ---> QDSMain.Exceptions.LogBucketErrorException: The Source Document was NOT reloaded successfully. DocumentPath=C:\QlikView_SourceDocs\xxxxxxxxxxx.qvw. || at QDSMain.ReloadTask.VerifyConditions(TaskResult taskResult) || at QDSMain.ReloadTask.Reload(String fileName, TaskResult taskResult, String sectionAccessUserName, String sectionAccessPassword, eReloadOptions reloadOption, String variableName, String variableValue, Boolean moniterCpuUsage) || --- End of inner exception stack trace --- || at QDSMain.ReloadTask.Reload(String fileName, TaskResult taskResult, String sectionAccessUserName, String sectionAccessPassword, eReloadOptions reloadOption, String variableName, String variableValue, Boolean moniterCpuUsage) || at QDSMain.DistributeTask.Execute(TaskResult currentTaskResult) || --- End of inner exception stack trace --- || at QDSMain.DistributeTask.Execute(TaskResult currentTaskResult) || at QDSMain.Task.AbstractTask.TaskExecution(ILogBucket logBucket, TaskResult taskResult)

      (2017-04-06 09:09:26) Information: Task Execute Duration=00:00:03.6788522

      (2017-04-06 09:09:26) Information: TaskResult.status=Finished

      (2017-04-06 09:09:26) Information: Notifying all triggers of new state:FinishedWithErrors

      (2017-04-06 09:09:26) Information: Notifying all triggers of new state:FinishedWithErrors - completed

      (2017-04-06 09:09:26) Information: Saving Task Result

        • Re: Reload with Section Access under different user fails
          Peter Cammaert

          The posted tasklog isn't very helpful. It simply says that the reload failed. Could be due to one or more of a hundred reasons. Check these:

           

          • Are there any error messages in the DocumentLog?
          • Is the Section Access user defined as ADMIN? I'm assuming that this is the case, or this user would not be able to reload a protected document in QV Desktop. But we need to be sure...
          • Is the Seciton Access user allowed to save the document in the target directory?
          • Which type of Section Access are you using? NTNAME or USERID/PASSWORD?
            • Re: Reload with Section Access under different user fails
              Carola Kofler

              Hi Peter,

              Thank you for the quick response.

              I did not have DocumentLog activated for this QVW, but tried that now, ran another reload but could not find any files. According to another forum post I found, those should be somewhere in C:\ProgramData\QlikTech\DistributionService\ or C:\ProgramData\QlikTech\DistributionService\1\Log but when I searched for the file, I did not get any results.

               

              To confirm:

              Section Access User is ADMIN and allowed to save the document in the target directory (Accesspoint I assume). We are using NTNAME for section access and have not had any issues in the 20+ other qvw files we have - which however are all reloaded with the standard service user.

               

              Additional info:

              as I have a close timeline, I have temporarily requested the standard service user to have read access to the restricted directory where all my data files are located. After attempting to run the task now - with Section Access on that other user set - I no longer get any error messages.

              After trying to remove data permissions for the standard service user and keeping the Section Access Reload on the other user, the reload fails again. So to me it looks like even if we attempt to reload a file with another user than the standard user - the standard user still must have access permissions to both data source and the file.

               

              If this is the case, we have a security risk that will potentially kill the project. I hope there is a solution for this.

                • Re: Reload with Section Access under different user fails
                  Miguel Angel Baeyens de Arce

                  The service user for QlikView services must be an admin and have all possible reduction field, if they exist on the section access, with all the possible values.

                   

                  This service account needs modify permissions on both the source folder where the QlikView Distribution Service reloads, as it overwrites the old file with the new information, and also writes the log if enabled, and the destination folder where the file is distributed or published.

                   

                  Easiest way to test is to log with the service account, open desktop, open the file and reload it and save it in the same folder where the QDS is reading.

                    • Re: Reload with Section Access under different user fails
                      Carola Kofler

                      Hi Miguel,

                       

                      the key requirement of the project is that the regular Qlikview service user cannot have access to data source or to section access.

                      Reason being that the credentials for the standard service user are known to a larger circle than can have access to this particular project.

                       

                      In the HELP menu I can see this information:

                      Section Access

                      By default, the reload will be performed as the user under which the QlikView Distribution Service (QDS) is running. The Section Access setting allows the use of another user when performing the reload. To by-pass the default setting, tick this check box and enter the desired user and password in the User Name and Password text boxes. To use the default setting, untick this check box.

                       

                      I attached a screenshot to show that I have defined that in my task - but according to the DocumentLog, Qlikview is still reloading the file with the standard service user.

                       

                      Note: the user I have been selecting for the time being is my own - and I have full permissions - just like the standard service user.

                       

                      I also saw your other reply to the document - do you think it is related to our section access being based on NTNAME - is it possible that overriding the standard service user only works with named user/password section access?

                        • Re: Reload with Section Access under different user fails
                          Miguel Angel Baeyens de Arce

                          I see your point here.

                           

                          The way QlikView works is using a user with enough privileges to write, read and interact with the operating system in different ways (creating files, writing permissions, reading config files, etc.). Among other things, this service account runs the QlikView software, and one of the functions is triggering reloads when they need to happen. So the user which starts and actually runs the reload process is the service account, and that cannot be changed without heavy third party development to impersonate one process as another user.

                           

                          However, section access allows you to achieve what you want: even if the service account will be the one reloading and saving the file, you can specify a pair of USERID/PASSWORD (which from the security standpoint, is way less secure than using an NTFS account) which this service account will use to open the application.

                           

                          Again, think of the service account as any other QlikView user. Now, let me use an example: I have an application with a section access using both NTNAME and USERID/PASSWORD. My username is DIR\MBAEYENS and has ADMIN access on the application. Your user, DIR\CKOFLER is not in the section access table.

                           

                          However, I want you to open the application to validate some charts and figures for me. While your user's NTNAME is not in my section access table, I give you this userid and password so you can open the app.

                           

                          Exactly the same is what happens with this section of the task in the reloading process. It is the "servicesmcrmeservice" account the one which opens the file, as in the example above us CKOFLER who opens the file. However, it will use the username and password to authenticate in the application.

                           

                          Now, this username and password MUST be specified hardcoded in the section access table, meaning it is strings of text for both the username and password.

                           

                          See the example below:

                           

                          Section Access;
                          
                          // Users from the company Active Directory
                          NTNamesTable:
                          LOAD * INLINE [
                          ACCESS, NTNAME, USERID, PASSWORD, COUNTRYCODE
                          ADMIN, DIR\MBAEYENS, *, *, *
                          USER, DIR\CKOFLER, *, *, ES
                          USER, DIR\CKOFLER, *, *, DE
                          ];
                          
                          
                          // user with full country permissions to reload, NOT in Active Directory
                          LOAD
                            'ADMIN' AS ACCESS
                          , '*' AS NTNAME
                          , 'RELOADER' AS USERID
                          , 'yxlJxRW85#dF9' AS PASSWORD
                          , COUNTRYCODE
                          FROM CountryCodes.qvd (qvd);
                          
                          Section Application;
                          

                           

                          When opening the application, security will apply as follows:

                          • User DIR\MBAEYENS is allowed to see ALL countries, and when opening the file locally, do whatever (ADMIN).
                          • User DIR\CKOFLER is allowed to see ONLY DE and ES as countries, and when opening the file locally, will be able to do some things as defined by the admin, but not all (like, no reload or no export)
                          • Any other user will be prompted to provide a user name and a password. If the credentials above are not used, the user will be denied access and will not be able to open the application, either locally or on the AccessPoint.

                           

                          Following this example, when the reload happens, user "VIZRTINT\servicemscrmeservice" will try to open the file, and unless the task has "RELOADER" as User Name and "yxlJxRW85#dF9" as the password, the reload will fail. Yet, the user who will perform the reload, open the file and save it will be "VIZRTINT\servicemscrmeservice" which has no access whatsoever to the information contained in the application.

                           

                          Nevertheless, this is not the most secure way to prevent anyone to read information from the application, as the password for the task must be stored somewhere. It is stored encrypted (I don't know the algorithm that Qlik uses) but still not a good idea if security is the utmost concern.

                           

                          Hope this makes it a bit clearer now.

                            • Re: Reload with Section Access under different user fails
                              Carola Kofler

                              Hi Miguel,

                               

                              Thank you for the detailed explanation. It makes sense now - even though it is not exactly what we wanted. I will flag the answer as "CORRECT" as I also managed to get the reload working following the example you provided. However I will check with the project owner if this is sufficient or if we will need to live without automatic reload and rather have all authorized persons reload the central qvw file via local Qlikview clients and NTNAME authentication. This will eliminate the need for the service user to have access to data source and file.

                                • Re: Reload with Section Access under different user fails
                                  Miguel Angel Baeyens de Arce

                                  Ideally, the service account is managed by IT who have strict NDAs in place and even if they have access to the password or the full application they cannot do anything with it.

                                   

                                  You can also hide all the script or at least, the section access part with QlikView Desktop, which means that you need a password to read and modify that part of the script. In addition, note that for ACCESS = USER, you can disable to edit the script.

                                   

                                  Perhaps, saving the QVPR folder into a database instead would provide a bit more secure environment, if only because everything is stored there and an additional user and password are required to read the information from the task, instead of an XML file. Still, you need to specify in the QMC the username and password for SQL Server (of course, unless you use Windows credentials but that would mean the same issue you have today).

                                   

                                  Impersonating (like when using RUNAS in the Windows command line) could be another case, but does not work out of the box in QlikView, where all services must run under the same account.

                                   

                                  A combination of a (partially) hidden script, strict USER section access, security settings in the document, and password required in the QMC for the task should be enough to prevent anyone accessing your data.

                                    • Re: Reload with Section Access under different user fails
                                      Carola Kofler

                                      Hi Miguel,

                                       

                                      You are right - we may have to rethink our current set-up in terms of service user management.

                                       

                                      The section access is already placed in a hidden script though - if I go for having the guys run the files individually I will move the entire script there.

                                       

                                      Having the source data in a database would have been nice - but in this project the participants accessing the data and handling updates/additions should only have limited access to their data range as well - looping through individual files that can be conveniently assembled by the project owner in a central place seemed like the smoothest process.

                                       

                                      Thanks again for all the assistance!

                        • Re: Reload with Section Access under different user fails
                          Andrew Whitfield

                          Hi Carola,

                           

                          the document log is created in the same folder as the actual qvw.

                           

                          Andy

                            • Re: Reload with Section Access under different user fails
                              Carola Kofler

                              Hi Andy,

                              thank you for confirming the path. To reduce the back-and-forth in this phase, I am storing the test files directly on SourceDoc. There I could not find a DocumentLog.txt file - but a couple of DOCUMENTNAME_TIMESTAMP.txt files - containing only my script from the data load.

                              I am attaching the version that would coincide with my attempt to remove the standard qlikview user from the permission table for section access.

                                • Re: Reload with Section Access under different user fails
                                  Bill Britt

                                  HI Carola,

                                   

                                  This log doesn't help. It is not the log when the document was ran.

                                   

                                  Does this happen on all documents that has section access applied?

                                  Is the service account that the distribution service is running under part in Section Access?

                                   

                                  Bill

                                    • Re: Reload with Section Access under different user fails
                                      Carola Kofler

                                      Hi Bill,

                                       

                                      I do not keep DocumentLogs normally - I only activated it for this specific qvw, then ran the reload task via the Management Console- and this was the file I found in the folder afterwards. If there are any other ways for me to provide a proper log file, please let me know. I already thought that a txt containing my script is not really what will help in this case.

                                       

                                      To answer your questions: I work with section access on lots of qvw files - they all keep the permission data in the same way. What's special with this qvw: the data security level of this is very high, so the goal is that users who have the password of the regular service user cannot log on to Qlikview and work with the file. My first approach was to work with the "Section Access" flag under the RELOAD tab of the task (which is what I have not done before), where I entered actually myself for test purposes as the other user. I have full permissions on Qlikview, the server, the restricted data area and the restricted file. This was not successful until I enabled data access to the standard service user and also included the standard service user in the permission file for the section access.

                                       

                                      The HELP page on this tab is not extremely extensive - it says: enter username and password - which is what I did. Exactly as it appears in the permission file for section access. So the problem could be that my expectation of "Section Access" under "Reload" is wrong.

                                       

                                      To confirm again: the goal would be to reload and publish a restricted document with a new service user, which is not the same user that runs all Qlikview services. The standard service user that usually operates Qlikview should not have access to the data source or the published document.

                                       

                                      So if you could as a next step confirm if my expectation of "Section Access" under "Reload" is realistic or not - I may have to close this question and open another one in a different group, to get advise on the core requirement.

                                        • Re: Reload with Section Access under different user fails
                                          Bill Britt

                                          What version of Qlik are you running?

                                          Does the account you are trying to use how the necessary rights to all the data?

                                          Enable the the logging in the QVW and then post the Document log after the reload with the task log.

                                          attach your SA header line

                                           

                                          ACCESS NTNAME >>>>

                                           

                                          Bill

                                            • Re: Reload with Section Access under different user fails
                                              Carola Kofler

                                              Hi Bill,

                                               

                                              The Qlikview version as well as a task log from a reload attempt when the error occurred is included in my initial question. And I enabled logging in QVD, produced a DocumentLog which is attached to my response to Andy further up.

                                               

                                              Section Access is working - I established that while manually reloading the document. Plus when giving the standard Qlikview service user read-access to the restricted directory as well as including it in the section access permissions, the reload task where I specify another user under "Section Access" in the "Reload" tab also works - however once we remove the standard Qlikview service user from the section access for that document, the reload task fails.

                                               

                                              Is "Section Access" under "Reload" supposed to replace any permissions of the standard Qlikview Service User? It does not look like it.

                                                • Re: Reload with Section Access under different user fails
                                                  Miguel Angel Baeyens de Arce

                                                  You do need to add either the service account running the QlikView services (more convenient specially if you are not using USERID/PASSWORD in the section access) or define a USERID/PASSWORD in each and every section access protected file and set those up in the QMC task.

                                                   

                                                  Note that what is happening behind the scenes, roughly speaking, is the service account opening the QVW, reloading it and saving it. If the user which opens the QVW does not have permissions or is not in the section access, the reload will obviously fail, as it would do if any user not in the section access table tries to open the app.

                                                   

                                                  That section in the QMC only replaces if the service account is not added to the section access table and you need to specify a USERID/PASSWORD combination as mentioned above. Otherwise, even not enabling the section access in the QMC will work, as the service account will be able to open and reload the file just normally.

                                                   

                                                  As for the script execution log files, they are enabled in the Document Settings and stored in the same folder than the QVW file is.

                                      • Re: Reload with Section Access under different user fails
                                        Andrew Whitfield

                                        DOCUMENTNAME_TIMESTAMP.txt file - that os the document log file ;-)