Issue with IndexAdd since moving to 1.7.1 [RESOLVED - internal script error]

[qliknerd]

Hi all

We've recently updated to 1.7.1 of the QDF and are having some issues with IndexAdd in some of our generators.

What's strange is that some files work and create indexes in the Shared Folder, whereas a number of them don't and are throwing errors with this part of the indexing: Call DoDir ('[Folder]\[File].qvd', '_tmp_DoDir','','','true')

Anyone else having any problems, or can offer any help?

Thanks

Ian

We've tested by reverting to the 5.DoDir.qvs and 12.Index.qvs from 1.6.0 and they create the index file successfully, so something is awry with the 1.7.1 version of these files.

[Magnus_Berg]

Hi Ian, could you validate if 5.DoDir and 12.Index that is stored under https://github.com/QlikDeploymentFramework/Qlik-Deployment-Framework/tree/master/3.Include/4.Sub works better? As they contains some minor fixes not released yet. Regards Magnus

[qliknerd]

Hi Magnus, thx for the reply. I tried the versions from GitHub and it results in the same error message.

Some more digging:

The script pulls in multiple csv files, then concatenates into a single QVD file. If there are no updates it just brings in the latest QVD file. The QVD is then stored before running IndexAdd.

If there are CSV files, then the Indexing works.

If it just loads the QVD then stores, then Indexing throws the error.

I can't see why this wouldn't work in the same way unless there are different headers when generating a QVD from scratch or when loading direct.

Any thoughts?

[Magnus_Berg]

Could you attach the log created during reload for me to analyze? Regards Magnus

[qliknerd]

09/08/2018 16:44:47: 0197  STORE [AP02] INTO 'E:\QlikView\SourceDocs\98.System\16.SOP\2.QVD\\AP02.qvd'

09/08/2018 16:44:47: 0198  Call IndexAdd('E:\QlikView\SourceDocs\98.System\16.SOP\2.QVD\AP02.qvd')

09/08/2018 16:44:47: 0025    if not right( trim('E:\QlikView\SourceDocs\98.System\16.SOP\2.QVD\AP02.qvd') , 4)= '.qvd'  then

09/08/2018 16:44:47: 0029    Call DoDir ('E:\QlikView\SourceDocs\98.System\16.SOP\2.QVD\AP02.qvd', '_tmp_DoDir','','','true')

09/08/2018 16:44:47:         Error: Script line error:

09/08/2018 16:44:48: 0031    let vL.QDF.DoDir = NoOfRows('_tmp_DoDir')

09/08/2018 16:44:48: 0033    for vL.QDF.LoopFromDoDir = 0 to vL.QDF.DoDir

09/08/2018 16:44:48:         Error: Script line error:

09/08/2018 16:44:49: 0034    LET vL.QDF.IndexQVD = peek('_tmp_DoDir.FullyQualifiedName',,'_tmp_DoDir')

09/08/2018 16:44:49:         Error: Script line error:

09/08/2018 16:44:50: 0035    LET vL.QDF.QVDFileName= peek('_tmp_DoDir.DoDirFileName',,'_tmp_DoDir')

09/08/2018 16:44:50:         Error: Script line error:

09/08/2018 16:44:51: 0037    call vL.FileExist ('')

09/08/2018 16:44:51: 0012      do while right( '' , 1 ) ='\'

09/08/2018 16:44:51: 0017      if IsNull(filesize('')) = -1  then

09/08/2018 16:44:51: 0018        SET vL.FileExist='false'

09/08/2018 16:44:51: 0019      else  

09/08/2018 16:44:51: 0022      set vL.QDF.Root=

09/08/2018 16:44:51: 0023    end sub

[Magnus_Berg]

Hi Ian, Please try latest version of 12.Index.qvs available on GitHub, where validation been enhanced, and get back to me on status.

/Best regards Magnus

https://github.com/QlikDeploymentFramework/Qlik-Deployment-Framework/blob/master/3.Include/4.Sub/12....

[qliknerd]

Hi Magnus thanks. Looks like the issue still remains, although there is only one error now rather than four:

10/08/2018 11:59:28: 0198  STORE [AP02] INTO 'E:\QlikView\SourceDocs\98.System\16.SOP\2.QVD\\AP02.qvd'

10/08/2018 11:59:28: 0199  Call IndexAdd('E:\QlikView\SourceDocs\98.System\16.SOP\2.QVD\AP02.qvd')

10/08/2018 11:59:28: 0025    if not right( trim('E:\QlikView\SourceDocs\98.System\16.SOP\2.QVD\AP02.qvd') , 4)= '.qvd'  then

10/08/2018 11:59:28: 0029    Call DoDir ('E:\QlikView\SourceDocs\98.System\16.SOP\2.QVD\AP02.qvd', '_tmp_DoDir','','','true')

10/08/2018 11:59:28:         Error: Script line error:

10/08/2018 11:59:30: 0031    let vL.QDF.DoDir = NoOfRows('_tmp_DoDir')

10/08/2018 11:59:30: 0034    if vL.QDF.DoDir > 0  then

10/08/2018 11:59:30: 0153    Set vL.QDF.AltIndexPath =

10/08/2018 11:59:30: 0154    Set vL.QDF.IndexQVD =

10/08/2018 11:59:30: 0155    set vL.QDF.DoDir=

10/08/2018 11:59:30: 0156    set vL.QDF.LoopFromDoDir=

10/08/2018 11:59:30: 0157    set vL.QDF.ContainerName=

10/08/2018 11:59:30: 0158    set vL.QDF.SharedConfigPath=

10/08/2018 11:59:30: 0159    SET vL.ContainerFolderName=

10/08/2018 11:59:30: 0160    SET vL.RootPath=

10/08/2018 11:59:30: 0161    SET vL.Comment=

10/08/2018 11:59:30: 0162    SET vL.QDF.ContainerPathName=

10/08/2018 11:59:30: 0163    Set vL.QDF.IndexFolderName =

10/08/2018 11:59:30: 0165  END SUB

10/08/2018 11:59:30: 0205  LET vL.after=now()

10/08/2018 11:59:30: 0206  LET vL.TableNumber=0+1

10/08/2018 11:59:30: 0207  let vL.TableName_1 ='AP02'

10/08/2018 11:59:30: 0208  let vL.NoOfRows_1 = NoOfRows('AP02')

10/08/2018 11:59:30: 0209  let vL.NoOfFields_1 = NoOfFields('AP02')

10/08/2018 11:59:30: 0210  let vL.TableTime_1 =time(Timestamp('10/08/2018 11:59:30') - timestamp('10/08/2018 11:59:28'),'hh:mm:ss')

10/08/2018 11:59:30: 0215  trace 'Dropped Table AP02'

10/08/2018 11:59:30: 0215  'Dropped Table AP02'

[Magnus_Berg]

Strange, do you get the error during Call DoDir or straight after? Cant reproduce the error Could you attach the complete erorlog or run a debug? Ive created a container under 98.systems named 16.SOP running below commands without any problems.

call LCGV('SOP','QVD');

Call IndexAdd ('$(vG.SOPQVDPath)AP02.qvd'); alternative

Call IndexAdd ('$(vG.SOPQVDPath)*');

Another idea is that its something "corrupted" in AP02.qvd meta data?

Have you validated that QlikView have write access under 99.Shared_Folders\5.Config\Index where the index is written?

Cheers

[qliknerd · Accepted solution]

Magnus thanks again for looking into this, I think I've found the issue. Within the main load script we are looping multiple folders for multiple csv files and had called the function DoDir. These are legacy qvd generators used prior to QDF.

What was happening was that DoDir is getting called from the QVD and looks like it is automatically using the local DoDir rather than the QDF version

Prefixing the "local" function with local.DoDir (I think we'll rename the whole thing for the avoidance of doubt!) resolves the issue.

Apologies for you spending time on this, but it is very much appreciated and actually helped us to fix a number of older scripts.

Many thanks and have a great weekend!

Ian