topic How to load multiple tables at the same time from db1 to db2 in Talend Studio

How to load multiple tables at the same time from db1 to db2

Anonymous — Wed, 17 May 2017 21:40:07 GMT

Hi,

I am taking over an ETL job that is moving data from Oracle to HPE Vertica in Talend 6.3.1 free edition. In this job there are about 12 different tables being moved. The vertica cluster has 4 nodes and can easily handle parallel processing. When I kick off the talend job to load the 12 tables, I see that the job is executing the tOracleInput task one table at a time. Can anyone tell me if there is a way I can kick off all 12 tasks at the same time? I'm fairly new to Talend and I can't seem to find anything in the talend forum that tell me how to do this. Thanks.

Re: How to load multiple tables at the same time from db1 to db2

TRF — Wed, 17 May 2017 22:24:22 GMT

Hi,

With the free edition you have 2 options:
1- have separated subjobs (in the same job) not linked one to each other + tick the option "Multi thread execution" in Extra tab of project
2- have 1 external subjob per table + tick the option "Multi thread execution" in Extra tab of project for parent and child jobs

If you choose 2, both operations "select from Oracle" and "insert into Vertical" should be realized in the same subjob. If you need to manage the order of insert operations, you need to get the result using a memory list or a file.
Think about tPreJob and tPostJob if you need some operations when the job starts or finish.

Hope this helps.

Re: How to load multiple tables at the same time from db1 to db2

Anonymous — Wed, 17 May 2017 22:44:47 GMT

Thank you! That is exactly what I was looking for.

Re: How to load multiple tables at the same time from db1 to db2

TRF — Wed, 17 May 2017 22:52:55 GMT

Great!
Feel free to come back if you need more help.