topic How to do Bulk Load for InfiniDB in Talend Studio

How to do Bulk Load for InfiniDB

Anonymous — Thu, 15 Nov 2012 04:05:08 GMT

Hi guys,
I'm having a hard time using Talend when inserting 100millions up to 300millions of data because of the "Heap memory" exception. And as a quick workaround, I wanted to use the cpimport utility or the bulk load of InfiniDB in my talend jobs.
Can anyone show me an example of this scenario? Even a simple job below will do
tBulkLoad--><cpimport>-->tMySQLOuput
Thanks so much!

Re: How to do Bulk Load for InfiniDB

Anonymous — Thu, 15 Nov 2012 07:25:39 GMT

Hi
Which component are you using for InfiniDB? tJDBCxxx component? As I seen, Talend does not provide proprietary components for InfiniDB.

Re: How to do Bulk Load for InfiniDB

Anonymous — Thu, 15 Nov 2012 07:28:32 GMT

Hi shong,
Actually, I'm thinking of a shell script that will run the bulk loader,maybe two scripts it. Is it possible?

Re: How to do Bulk Load for InfiniDB

Anonymous — Fri, 16 Nov 2012 04:23:53 GMT

Hi shong,
Actually, I'm thinking of a shell script that will run the bulk loader,maybe two scripts it. Is it possible?

I know little about InfiniDB database, however, if the shell script can be called and executed from command prompt, you can use tSystem component to execute the script in Talend.