Downloading files and uploading into tables

[imholzj]

I have 15 files that need to be downloaded and then loaded into database tables.  The file names don't match the table names.

It seems like I should create a hash so I used tJava to:

   java.util.Map<String, String> files2tables = new java.util.HashMap<String, String>();

   files2tables.put("file1", "Table1");
   files2tables.put("file2", "Table2");
   files2tables.put("file3", "Table3");

Put this into globalMap:

   globalMap.put("files2tables", files2tables);

Then an array for just the filenames (keys):

   globalMap.put("files", files2tables.keySet());

Then I wanted to tForeach, but I can't seem to get the list of files to process into the tForeach.

Is there a better way to approach this problem?

jji

[David_Beaty]

Hi,

You can create the list of files using a tFixedFlow component (if they really are fixed file names to table names), and then connect in a tFlowToIterate.

This will loop through each file/table name pair and allow you to read it in and load it into a table. You havent mentioned whether they are the same schema, so I've assumed they are.

[brama]

Hi,
My way of doing these is creating a static lookup table in db with filenames and equivalent table names. Take the count of files and use it for the loop. Then in each iteration read the file name from table and its equivalent table name store it in context, then use this context values to insert the data from file to table.
Thanks,
Ramachandran B

[imholzj]

Thanks, that's working.

If I want to wait until I download all the files before I continue, what do I test for?  Does the tFlowToIterate set a global var when done?

jji

[David_Beaty]

I think you'll probably need to upload a screenshot of your job so far for anyone to advise further.

[imholzj]

I'd like to download all my files before I move on to loading them into the DB.

Is there a way to complete this IF condition with the size of the tFixedFlowInput_1 Array?

Or is there another "Talend way" of doing this?

jji