Parse text (RegEx ParseAll ?)

[xyz_1011]

Hi all,

i have a regular Qlik script file. Among other script content, the script also includes multiple $(Must_Includes) statements.

Within one of my automations, i am receiving this file and i managed to get to its content (basically a long string). From this string I want to

1. extract all occurances of include statements in a given file and and create a list of include statements - i.e., like this:  $(Must_Include=$(vModulesRoot)SALES\Revenue.qvs);, $(Must_Include=$(vModulesRoot)SALES\Customer.qvs);
   Would i be able to do this with a regex() ? Or any other way in an automation ?
2. Once i have extracted all includes and build a list of them, i would like to loop over this list of include statements and
   1.  get the filename (following the above example: Revenue.qvs and Customer.qvs) of every include statement and
   2. next to it: also the directory (following the above example: SALES)

Any advise on how to get there wohld be greatly apreciated!

Thanks in advance!

[AfeefaTk]

Hi @xyz_1011 

Let me try to understand the workflow.

So you are getting the script content using the Get Load Script block and want to get all the include statements from the script, add them to the list and loop over this list, get the filename and folder name from each item.

Do you have the automation on which you are working?

Thanks

[xyz_1011]

Hey @AfeefaTk - i believe outlining the whole use case would be something for a dedicated call (which I'd be happy to jump on if you like 🙂 ) rather than this forum post. What i am after is part of a complex use case. But let me try to illustrate with the attached (simplified) example automation as well as the following example description:

• I have a script (QVS file which i am retreiving from a git lab repo). The script looks something like this (might vary from script to script):
  
  $(Must_Include=$(vModulesRoot)03_SALES\MOD_Revenue.qvs);
  call StoreIntoSDL ('MOD', 'Revenue', '03_SALES', 'Revenue');
  call CreateMetaData ('Revenue', 'GEN_MOD_Revenue.qvs', 'MOD_Revenue.qvs', '03_SALES');
  
  
  //=========================================================================================
  // Trace ********* started...
  //=========================================================================================
  $(Must_Include=$(vModulesRoot)04_MARKETING\MOD_Campaign.qvs);
  call StoreIntoSDL ('MOD', 'Campaign', '04_MARKETING', 'Campaign');
  call CreateMetaData ('Campaign', 'GEN_MOD_Campaign.qvs', 'MOD_Campaign.qvs', '04_MARKETING');
  
  
• I base decode the QVS file to get the content as a string
  
  
• From this string i want to extract all include statements. To be specific, i want to create another string (iedeally a list) with every extracted include statement, which looks like this (every item represents a path in a gitlab repo): 
  
  04_MARKETING\MOD_Campaign.qvs
  03_SALES\MOD_Revenue.qvs
  
  
• After that i want to iterate over every item i built in the previous step and use a GitLab Get File From Repository to fetch the file...
  
  

I hope, what i am after became a bit clearer...in essence, what i am struggling with is: Extract the substring from every include and then iterate over every extraction.

Thanks a lot upfron!

[blaise]

Im far from a regexp expert but rather good a googling 🙂 so what about apply a regExp parse all function on the string:

(?<=Must_Include=)(.*)(?=;)

I tested it with some examples in a variable (vString) with the output block with the following code (formula parsing off):

{regexparseall: {$.vString}, '(?<=Must_Include=)(.*)(?=;)'}

The output will not include Must_Include and the semi-colon, the regExp takes the string between these two strings.  It also includes a ) at the end as the regexp fails if I use ); instead of ; as the end-string.

[AfeefaTk]

Hi @xyz_1011 

This formula works 

{regexparseall: {$.vString}, '(?<=Must_Include=)(.*)(?=;)'}

But you also need to remove this $(vModulesRoot) from the path right?

[AfeefaTk · Accepted solution]

Hi @xyz_1011 

I have tried this using custom code block.

Please take a look at the json file and let me know if that works for you.

Thanks

[AfeefaTk]

Hi @xyz_1011 

I have also done this using regex parse all formula.

Hope this works.

Thanks

[xyz_1011]

Hey @AfeefaTk  Very elegant! I had no clue that there is a way to work with PHP. This is cool! 

(obviously, it also does the job 😉 Thanks!!)

[AfeefaTk]

Hi @xyz_1011 

I would suggest you try my recent automation which uses the regex formula by replacing the custom code block. This should be a more clean solution than the custom code one.

Thanks

[xyz_1011]

Hey @AfeefaTk  thanks again for your help. I am running into an issue though, when trying to get a file from my repo after parsing the includes:

1.  storing every extracted & parsed include ina variable:

2. trying to get a file from my repo using this variable:

3. Runn ing into this error, where the / seems to cannot be decoded properly:

Any idea how to solve this ?