topic Load multiple csv files with different structures into postgre sql tables in Talend Studio

Load multiple csv files with different structures into postgre sql tables

Sindhura1 — Tue, 27 Jul 2021 13:38:44 GMT

I have csv files with different structures in my folder and I need to load them into target postgresql tables.

Can anyone please help me with this?

gjeremy1617088143 — Tue, 27 Jul 2021 13:47:55 GMT

Hi, @Sindhura V S

do you use a studio edition or an open source ?

Do you know how many different structures you have ?

Send me love and kudos

Sindhura1 — Tue, 27 Jul 2021 14:12:28 GMT

I use enterprise edition. I have 9 files with 9 different schema structures.

gjeremy1617088143 — Tue, 27 Jul 2021 14:19:11 GMT

In entreprise edition dynamic schema can be use to read csv with not defined structure but they have to had header

the only problem is you can't set key for database, if you just want to insert with an autoincrement it could works.

but you need to know in wich table you want to insert before i presume ?

Sindhura1 — Tue, 27 Jul 2021 14:24:14 GMT

I tried using dynamic schema but the column data types are not handled correctly.

gjeremy1617088143 — Tue, 27 Jul 2021 14:33:58 GMT

maybe this article can help you :

it's a lot of code but it permit to reatribute dbtype from the database, the source is a database but it can also work with csv :

https://www.talend.com/blog/2019/11/11/migrate-data-between-databases-one-job-using-dynamic-schema/