Skip to main content
Announcements
Global Transformation Awards! Applications are now open. Submit Entry
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

loop through to load all files from a folder and its subfolders?

Hi all,
This is my folder structure
MainFolder
       20150102
                      File1a.txt
                      File2ab.txt
                      File3.txt
       20150103
                      File1ab.txt
                      File2a.txt
                      File3ab.txt
                      File4.txt
       20150104
                      File3a.txt
                      File5.txt
                      File6.txt
I'm trying to use a Loop function but each text file has a different parameter (comma, space, dash, etc. ). I'm also trying to use a wildcard.
FROM

(
txt, codepage is 1252, no labels, delimiter is '|', msq);
my loop statement is written similar to this one but fot TXT file instead of CSV.
how do I combine different parameter, scan folders and subfolder, HELP!!
Thanks in advance
1 Solution

Accepted Solutions
simospa
Partner - Specialist
Partner - Specialist

But do you know which files have a specific separator? Can you identify them by the filename?

S.

View solution in original post

6 Replies
Not applicable
Author

Not applicable
Author

I used that post as a guide but I have specific folder name that I need to search by.


set Concatenate=;     
// subroutine
sub ScanFolder(Root)
for each Ext in 'File1a.txt' // use this to identify file names to load but I can't use the wildcard to pull all File1*
for each FoundFile in filelist(Root & '\*' & Ext)
FileList:     

$(Concatenate)
load *
from
[$(FoundFile)](txt, utf8, no labels, delimiter is '\t', msq, no eof); // my load has different parameters depending on the file name.

set Concatenate = concatenate;
next FoundFile
next Ext

for each SubDirectory in dirlist(Root&'\*')
call ScanFolder(SubDirectory)
next SubDirectory
end Sub

C
all ScanFolder('C:\');

simospa
Partner - Specialist
Partner - Specialist

But do you know which file has a specific separator? For example: *1*.txt has comma, *2* has slash, etc...?

You could try make several loop, one for type of separator

set Concatenate=;     
// subroutine
sub ScanFolder(Root)
for each Ext in 'File1a.txt' // use this to identify file names to load but I can't use the wildcard to pull all File1*
for each FoundFile in filelist(Root & '\*' & Ext)
FileList:     

$(Concatenate)
load *
from
[$(FoundFile)](txt, utf8, no labels, delimiter is '\t', msq, no eof); // my load has different parameters depending on the file name.

set Concatenate = concatenate;
next FoundFile
next Ext

for each SubDirectory in dirlist(Root&'\*')
call ScanFolder(SubDirectory)
next SubDirectory
end Sub


sub ScanFolder_2(Root)
for each Ext in 'File1a.txt' // use this to identify file names to load but I can't use the wildcard to pull all File1*
for each FoundFile in filelist(Root & '\*' & Ext)
FileList:     

$(Concatenate)
load *
from
[$(FoundFile)](txt, utf8, no labels, delimiter is ',', msq, no eof); // my load has different parameters depending on the file name.

set Concatenate = concatenate;
next FoundFile
next Ext

for each SubDirectory in dirlist(Root&'\*')
call ScanFolder(SubDirectory)
next SubDirectory
end Sub


C
all ScanFolder('C:\');

Call ScanFolder_2('C:\');


etc...


S.

Not applicable
Author

Yes.
I'm trying this but only one Loop is working.

simospa
Partner - Specialist
Partner - Specialist

But do you know which files have a specific separator? Can you identify them by the filename?

S.

Not applicable
Author

yes. I have all specific separator.