topic Re: Qlikview concurrent extraction in QlikView

Qlikview concurrent extraction

nsm1234567 — Thu, 15 Oct 2015 08:35:41 GMT

Hi There,

I'm hoping someone can assist with an issue I'm having. The scenario is that data is being stored per week in a number of QVD's. This ensures that, every day, a relatively small number of weeks need to be extracted. A problem arises though when changes are made to the extractor which results in the entire history needing to be revised. Currently, there is a very linear process where the script loops through each week, extracts the data for that week and then stores it in a QVD. What I'm wondering is if it's possible for QV to run multiple weeks concurrently as I've outlined in the attached illustration. One option I've considered is opening multiple instances of the extractor app and running each for a set number of weeks, but this wouldn't be an ideal situation. Is what I've outlined in the attached doable?

Re: Qlikview concurrent extraction

Gysbert_Wassenaar — Thu, 15 Oct 2015 10:37:37 GMT

Not as far as I know. One qv.exe instance can only execute one script at a time and only sequentially.

Re: Qlikview concurrent extraction

petter — Thu, 15 Oct 2015 11:49:32 GMT

If you run the load scripts on a QlikView Server either with or without Publisher you can create multiple tasks that start at the same time. Then you will get your cont extraction.

If you want to achieve the same with desktop which I suspect that your are not ... you can even here do a concurrent extract by starting each extract as separate QV.EXE /R extractor<n>.QVW in a Windows CMD-file by spawning each QV.EXE with the Windows command START.EXE.

Whether you will gain a lot depends on a number of factors - but you should be able to draw siginificant improvement in general.

As Gysbert said there is no parallelism inside a single load script. So you have to possilbly refactor you script into multiple independently executable scripts.

Re: Qlikview concurrent extraction

nsm1234567 — Fri, 16 Oct 2015 08:34:27 GMT

Hi Peter,

Thanks for the response, the note on using START.EXE was particularly helpful. I've ended up using the below script which does work. The one issue I had was that reload and then save is problematic when you're using a single file. I resolved by leaving the file open and closing the file with a macro in the "OnPostReload" action. My only concern is that something like this would be difficult to implement (we have Server, but it isn't used for the file reloads, so I'm using a .bat file to kick off the process).

let MyfileDirectory = DocumentPath();

if len('$(MyVar)') = 0 then

for each file in FileList('MYFILENAME*')

LET fileName = TextBetween('$(file)','MYFILENAME','.');

execute cmd.exe /C "start "c:\Program Files\QlikView\" "QV.exe" /l /vMyVar=$(fileName) "$(MyfileDirectory)"";

NEXT file

ENDIF

if len('$(MyVar)') <> 0 then

TABLE:

LOAD *

FROM

[MYFILENAME$(MyVar).QVD]

(qvd);

store * from TABLE into $(MyVar).qvd (qvd);

ENDIF