Loading 200+ Files to 200+ Snowflake tables

[Rishab1]

Hi , I have a problem in which i want to load 200+ files into 200+snowflake tables (upsert logic) , Earlier for 50 around tables i had used 50 different filelist and 50 fileinputdelimited components . Are there any inbuilt components to handle this or a mechanism which could be used to implement this . I don't want to use filelist and fileinputdelimited for every new File which we get.

Some of the files coming have similar names to differentiate them i have used regex.

Is there a possibility to create a hashmap and use that in the single component to load all the files to respective tables and implement upsert logic.

[Anonymous]

Hi

To iterate multiple files in the specified directory, tFIleList is the right component to do it. If some files are not updated, use the tFilelist to iterate all files and then select only the new files and process these new files?

Regards

Shong