5 Replies Latest reply: Dec 29, 2014 7:25 AM by Yakir Manor RSS

    QlikView Server and Partial Reloads

    Matt Clark

      Are user-driven partial reloads allowed from QV Server?

      I have allowed macro executions on server (also unsafe macro executions), allow full and partial reloads in the document security tab. I see that there is no action associated with full or partial reloads, so I wrote a macro that calls Partial Reload like this:

       


      Sub PartialReload
      ActiveDocument.PartialReload
      End Sub


      This procedure works as expected in QV Desktop, but it does not work in QV Server running zero-footprint ajax. I have also tried the DoReload and ReloadEx macro calls without success.

      Are partial reloads not working in QV Server? Does anyone have an example of a partial reload working? I can create a sample .qvw document illustrating this problem if needed.

      This partial reload is needed because we have an application that is extremely complex using aggr() functions and we need to pre-process many operations after changing some settings. Our end-users would not be happy sitting for 60-240 seconds for a simple selection change, whereas pre-processing some calculations brings the application into the usable range of 3-10 seconds. The data sets are currently not huge (~.5M records), and we need a scalable solution for when we use millions of records.

      Matt Clark

       

        • QlikView Server and Partial Reloads
          Rob Wunderlich

          I don't think any reload API call will work in the Server environment. I believe the way this is usually done is make an http EDX request to Publisher to run the reload task. In the reload task Partial reload can be specified. The http url can be the launch action of a button.

          -Rob

            • QlikView Server and Partial Reloads
              Matt Clark

              Rob, THANK YOU for your reply, and I had a suspicion that was going to be the case. I'll see if I can get an example of the publisher request working.

              - Matt

                • QlikView Server and Partial Reloads
                  Linda Monincx

                  Hi Matthew,

                  I am facing the same issue that i need to kickoff a partial reload through the IE-plugin, did you manage to get it to work on a small business edition server?

                  Regards,

                  Linda

                    • QlikView Server and Partial Reloads
                      Matt Clark

                      Hi Linda,

                      My solution was to create an "EDX" event-driven task (as Rob suggested). Our license is an OEM version, so I am not sure if EDX is supported in the small business edition.

                      My QlikView document has a button that calls a macro function. This macro function sends an HTTP GET request (using WinHttp.WinHttpRequest.5.1 object) to a web page on the same server. The business logic on the web page requests a session key from the QV server, and then sends an EDX request to the server with arguments for the task name, the session key, and the password stored with the EDX.

                      The only thing missing from this process is a progress bar or method to "push" the just-reloaded data to the Ajax client. Our instructions tell the user to click around after a minute until the new data appears. If you have a better solution (or a progress bar!), I'd be grateful. Note that I cannot ge the "Allow dynamic data update" checkbox to stay on in the QV server setup page, so I suspect my own license is insufficient.

                      Matt

                • Re: QlikView Server and Partial Reloads
                  Yakir Manor

                  Hi,

                  this is for partial reload without the publisher

                  ill assume you know how to use the API,
                  it helps to search the 'QMS API Documentation', you can find it here -

                  QMS API Documentation - Version 11

                   

                  in the api if you search 'partial' you get 'TaskReloadMode Enumeration' so from that we understand there might be a chance (-:.

                   

                  lets look at 'C:\ProgramData\QlikTech\DistributionService\Tasks' there we can see the xml of all the tasks, and we can also find the task ID so we can delete the task or run it

                   

                  creating partial reload starting from a point where we have no tasks at all (you can have tasks but just for this example):

                  1. create the first task by going to: QMC -> Documents -> select the document we want to partial reload (development.qvw) -> Reload tab -> choose any: i did: Daily -> Apply

                  a file was created in the task folder with the name: 'Task_39614ba7-0628-4264-ac31-b553ba3a0862.xml'

                   

                  2. go to: Status -> Tasks -> verify a task was indeed created 'Reload of Development.qvw'

                   

                  3. open visual studio and run the following code: (some changes is needed to adjust for your file, here we want to apply it to 'Development.qvd')

                   

                  
                  // Initiate backend client
                  IQMS apiClient = new QMSClient();
                  
                  // Get a time limited service key
                  ServiceKeyClientMessageInspector.ServiceKey = apiClient.GetTimeLimitedServiceKey();
                  
                  // Get QlikView Server
                  ServiceInfo qds = apiClient.GetServices(ServiceTypes.QlikViewServer).FirstOrDefault();
                  DocumentNode document = apiClient.GetUserDocuments(qds.ID).Where(x => x.Name.Equals("Development.qvw", StringComparison.OrdinalIgnoreCase)).FirstOrDefault(); //Get first source document for this example
                  
                  
                  DocumentTask documentTask = new DocumentTask();
                  documentTask.QDSID = qds.ID;
                  documentTask.Document = document;
                  documentTask.Reload = new DocumentTask.TaskReload();
                  documentTask.Reload.Mode = TaskReloadMode.Partial;
                  
                  
                  documentTask.General = new DocumentTask.TaskGeneral();
                  documentTask.General.Enabled = true;
                  documentTask.General.TaskName = "Partial Reload of Development.qvw";
                  documentTask.General.TaskDescription = "Creating a task via QMS API";
                  documentTask.Scope = DocumentTaskScope.General;
                  
                  
                  Guid triggerID = Guid.NewGuid();
                  
                  
                  // Create a scheduled trigger
                  ScheduleTrigger st = new ScheduleTrigger();
                  st.ID = triggerID;
                  st.Type = TaskTriggerType.OnceTrigger;
                  st.Enabled = true;
                  st.StartAt = DateTime.Now;
                  
                  
                  // Create a list of trigger and add the scheduled trigger
                  List<Trigger> List_Trigger = new List<Trigger>();
                  List_Trigger.Add(st);
                  
                  
                  documentTask.Triggering = new DocumentTask.TaskTriggering();
                  documentTask.Triggering.Triggers = List_Trigger;
                  
                  
                  documentTask.Triggering.TaskDependencies = new List<TaskInfo>();
                  documentTask.Triggering.ExecutionAttempts = 1;
                  documentTask.Triggering.ExecutionTimeout = 0;
                  
                  
                  // Scope plays an important role
                  documentTask.Scope = DocumentTaskScope.General | DocumentTaskScope.Triggering;
                  
                  
                  //Create task
                  apiClient.SaveDocumentTask(documentTask);
                  
                  

                   

                  4. see the task 'Partial Reload of Development.qvw' was created, try running it - NOTHING HAPPENS!, how do we know, because we opened the 'Development.qvw' file in the qlikview client and did Settings -> Document Settings -> General tab -> Generate Logfile & Timestamp in Logfile Name, and no log file was created )-:, now lets run the the 'Reload of Development.qvw', we can see a log file was created,

                  also a new file was created in tasks 'Task_6c92c573-a395-47e2-8e75-a15f0d128b46.xml'.

                   

                  5. now here is where the magic happens:

                  a. open both tasks xmls in notepad++ (from 'C:\ProgramData\QlikTech\DistributionService\Tasks')

                  b. make sure you opened notepad++ using 'run as administrator'

                  c. now copy all the data from 'Reload of Development'  ('Task_39614ba7-0628-4264-ac31-b553ba3a0862.xml' if we open both xmls the names are inside) to the 'Partial Reload of Development.qvw' ('Task_6c92c573-a395-47e2-8e75-a15f0d128b46.xml') and save the file EXCEPT 'DistributeTask' open node, so basically all the lines except the first two.

                  d. restart the 'QlikView Distribution Service' from Start -> Administrative Tools -> Services

                  e. now run the the task - VOILÀ the task is running!, problem is, its doing a full reload )-:

                  f. open the xml again of the 'Partial Reload of Development.qvw' search for 'LoadType' in the 'SourceDocument' node and change it to 'PartialReload', restart the 'QlikView Distribution Service' again, run the task from the qmc.

                  g. if you dont want the trigger in the xml you can remove it otherwise the partial task will run with the trigger settings, this is the line:

                   

                  <ScheduleTrigger ID="0afcc688-06c2-4925-93ac-d153a298cb5d" Enabled="True" EnableDateTime="2014-01-01 00:00:00Z" ExpireDateTime="9999-12-31 23:59:59Z" RecurrenceType="RecurrenceDaily" RepeatEvery="1" HourStart="2014-01-01 02:00:00Z" DayStart="0" RepeatMaxCount="0" />
                  

                   

                  that's it! you have partial reload

                   

                  you might want to delete some tasks, copy the task ID (name of task is inside the file) and run the code:

                   

                  IList<TaskInfo> taskList, docTaskList;
                  List<string> taskstoremove = new List<string>();
                  taskstoremove.Add("6c92c573-a395-47e2-8e75-a15f0d128b46");
                  foreach (var item in taskstoremove)
                  {
                      try
                      {
                          var taskID = new Guid(item);
                          var task = apiClient.GetTask(taskID);
                          apiClient.DeleteTask(taskID, TaskType.DocumentTask);
                      }
                      catch (System.Exception ex)
                      {
                          response.Append(ex.Message);
                      }
                  }
                  

                   

                  you might want to run the task, this is no problem, create any app, console app or .net app what ever you want and run the code:

                   

                  apiClient.RunTask(new Guid("6c92c573-a395-47e2-8e75-a15f0d128b46"));
                  

                   

                   

                  CAUTION! CAUTION! CAUTION!

                   

                  do not delete any of the xml's (from 'C:\ProgramData\QlikTech\DistributionService\Tasks') deleting them without first delete them from code using 'DeleteTask' will crush 'QlikView Distribution Service' on restart and you wont be able to start the service again after stopping it!!!

                   

                  hope this helped!.

                   

                  im not sure this is the only solution, we don't have the Publisher licence so there might be other solutions using the publisher.

                  im not sure what happens if you delete the original task the 'Reload of Development.qvd' im too lazy to test it

                   

                  P.S.:
                  for some reason i just cant get any of the api methods to return all tasks,

                  iv tried the api apiClient.GetTasks and the apiClient.GetTasksForDocument while looping over all docs, nothing worked.

                   

                  maybe it's because we don't have publisher, finally iv found a work around:

                   

                  // Initiate backend client
                  IQMS apiClient = new QMSClient();
                  
                  
                  // Get QlikView Server
                  ServiceKeyClientMessageInspector.ServiceKey = apiClient.GetTimeLimitedServiceKey();
                  
                  
                  var serviceList = apiClient.GetServices(ServiceTypes.QlikViewServer);
                  
                  
                  List<TaskInfo> taskInfoList = new List<TaskInfo>();
                  TaskStatusFilter taskStatusFilter = new TaskStatusFilter();
                  taskStatusFilter.TaskStatuses = new List<TaskStatusValue>();
                  taskStatusFilter.TaskStatuses.Add(TaskStatusValue.Aborting);
                  taskStatusFilter.TaskStatuses.Add(TaskStatusValue.Completed);
                  taskStatusFilter.TaskStatuses.Add(TaskStatusValue.Failed);
                  taskStatusFilter.TaskStatuses.Add(TaskStatusValue.Running);
                  taskStatusFilter.TaskStatuses.Add(TaskStatusValue.Waiting);
                  taskStatusFilter.TaskStatuses.Add(TaskStatusValue.Warning);
                  List<TaskStatus> taskStatuses = apiClient.GetTaskStatuses(taskStatusFilter, TaskStatusScope.Extended); // A service call.
                  foreach (var taskStatus in taskStatuses)
                  {
                      taskInfoList.Add(apiClient.GetTask(taskStatus.TaskID));
                  }
                  

                   

                  from here you can easily run task by name or what ever. (-:
                  iv attached the project solution and the xmls.

                   

                  a good way to run the code is to create a console application (EXE) and run it with button macro that runs that application, in this example i compiled a small program that runs the partial reload, it searches for task with the name 'partial' on the 'development' document.


                  Sub ScriptPartialReload
                    set v = ActiveDocument.GetVariable("vLastPartialReloadAttempt")
                    v.SetContent now(),true
                    rem msgbox("Partial")
                    set v = ActiveDocument.GetVariable("vFileName")
                    vFile = v.GetContent.String
                      Platform = ActiveDocument.Evaluate("ClientPlatform()")
                      if isnull(Platform) or Len(Platform)=0 then
                    ActiveDocument.PartialReload
                      else
                    Set shell = CreateObject("WScript.Shell")
                    shell.Run "C:\QlikViewTasks\qv-user-manager.exe -t partial -d " + vFile
                      end if
                  End Sub
                  

                   

                  this will only work if you enabling the Allow macro execution on server and Allow unsafe macro execution on server on your Qlikview Server Managment Console.


                  Go to System->Setup->Qlikview Servers->Security tab.

                   

                  you might also need to enable macros in the document

                  Go to Tools -> Edit Module -> Requested Module Security: System Access, Current Local Security: Allow System Access

                   

                  the final result is:

                  Capture.PNG

                  adding in the end of the load script this part unable you to create a smart behavior for the reload button

                   

                  IF(IsPartialReload()) THEN
                    LET vLastPartialReload = Now();
                  ELSE
                    LET vLastReload = Now();
                  ENDIF
                  

                   

                  in the reload button properties in 'Enable Condition' write

                   

                  =Interval(vLastPartialReload - vLastPartialReloadAttempt, 'ss') > 0
                  

                   

                  and now the button will be disabled until the end of the reload process.

                   

                  Please

                  If you found it helpful (-:.