loading multiple files in a directory

[#1]

Hi, I'm trying to load multiple files from a directory which is created in the last 15 minutes. Each time i load a file, I inner join it to another file. At the end of the loop I wanted to store the whole table into a text file. The problem i am facing is that every time the script loops through a file, a new table is created.(ie a table named tempTable is created in the first loop, tempTable-1 is created in the second loop, and tempTable3 is created in the third loop) Is there a way to load them into the same table since all of them have the same fiels. My code is as below.

SET dir="D:\";
LET prevProcessTime=Timestamp(now()-makeTime(00,15,00));

for each file in filelist(dir&'gtp_start_loc*.csv')
LET fileCreationTime=filetime('$(file)');
if fileCreationTime>prevProcessTime then

tempTable:

LOAD gtp:start_time,
     gtp:usi,
     gtp:msisdn as key,
     
FROM
$(file)
(txt, codepage is 1252, embedded labels, delimiter is ',', msq);

inner join

LOAD Service,
     65&[Service Number] as key,
     
FROM
D:\crm.csv
(txt, codepage is 1252, embedded labels, delimiter is ',', msq);
endif
next file

store tempTable into abc.csv(txt);

[#2 · Accepted solution]

Please can you try this:

i feel that you should close the loop before the inner join.....

Not sure though. you can give it a try.

SET dir="D:\";
LET prevProcessTime=Timestamp(now()-makeTime(00,15,00));

for each file in filelist(dir&'gtp_start_loc*.csv')
LET fileCreationTime=filetime('$(file)');
if fileCreationTime>prevProcessTime then

tempTable:

LOAD gtp:start_time,
     gtp:usi,
     gtp:msisdn as key,
     
FROM
$(file)
(txt, codepage is 1252, embedded labels, delimiter is ',', msq);

endif

next file

inner join

LOAD Service,
     65&[Service Number] as key,
     
FROM
D:\crm.csv
(txt, codepage is 1252, embedded labels, delimiter is ',', msq);

store tempTable into abc.csv(txt);

Thanks,

Robinson

[#3]

Thanks Robinson, it work:)

[#4]

Gr8,.. U R Most Welcome

THanks,

Robinson