Read data from file and use SQL in a loop to DBInput

cmccarthy1676460227 — Fri, 15 Nov 2024 22:05:54 GMT

Can I get ideas, the scenario is

list of data in a CSV file, like name, database, tablename, frequency

would like to loop trough list of tables from file and pass to dBInput pipeline - can't use tablelist as case sensitive

thanks in advance

Re: Read data from file and use SQL in a loop to DBInput

Anonymous — Fri, 17 Feb 2023 03:16:16 GMT

You can use tFlowToIterate to iterate each row of data read from file, eg:

tFileInputDelimimited--main(row1)--tFlowToIteratate--iterate--iterate--tDBInput--main--tLogRow

on tDBInput: use the current tablename in the query, eg:

"select id, name from "+(String)globalMap.get("row1.tablename")

About usage of tFlowToIterate component, please read the component documentation.

if the tables have different structure, you need to use the dynamic schema.

Regards

Shong