Skip to main content
Announcements
See what Drew Clarke has to say about the Qlik Talend Cloud launch! READ THE BLOG
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

search inside folders for file load

Hi all,

I need to load files from sub folders of sub folders.

I have the syntax for the regular search:

  1. for each SubDirectory in dirlist( Root & '\*'
  2.      if(substringcount('$(SubDirectory)' & '@', '$(vRunList)') then 
  3.           call ScanFolder(SubDirectory) 
  4.      end if 
  5. next SubDirectory


but the files aren't in the SubDirectory - they are in the sub directories of it (multiple directories).

in each sub-sub directory I have files with the same name.

I need to load all this files and concatenate them into one table.


can somebody help me with the script syntax?



Thanks


Boris 

14 Replies
Gysbert_Wassenaar

You could add a parameter to the sub function for the table name so it will take two arguments: a start directory and a name for the table that will be created.


talk is cheap, supply exceeds demand
Not applicable
Author

Hi,

Try This:

SUB DoDir (Root)

FOR each File in filelist(Root& '\*.xls')

Raw_Data_New:

LOAD

*

FROM [$(File)]

(biff, no labels, table is Sheet1$)

;

NEXT File

FOR each Dir in dirlist (Root&'\*')

CALL DoDir(Dir)

NEXT Dir

END SUB

CALL DoDir('C:\Users\AGoel\Desktop\Shared Apps\Voter List\ERoll')

STORE Raw_Data_New into Raw_Data_New.qvd;

DROP table Raw_Data_New;

Not applicable
Author

So, Just to understand.

This is my Path: d:/temp/#run_id/#part_id/table_name_X.csv

root = d:/temp

my run_id is the Dir in Dir List, am I right?

and the part_id is transparent, it will load all files in the run_id directory?

Now,  one last issue, the table names in each part id are the same, I need to concatenate all the tables with same name. as I understand this script, it will create multiple tables, and because we have same named files it will

overwrite the tables, how can I overcome this?


Thanks a lot!!!


Boris

Not applicable
Author

but the file name should be the same as the parameter I will pass.

how can I check it during the loop?

Not applicable
Author

filename doesn't work in regular script.