Problems to connect Informix

[Anonymous]

Hello!

I would like help with a really big problem that I am facing right now, I have load data from an Informix database but Talend can't load more than 70 rows per second, if I try to load in this same database from Excel or Qlik I can read it at least 8 x faster.

I believe the problem occurs when Talend converts the data format from Informix to Talend.

If I try to connect just integer fields Talend performs with more than 4000 rows/sec but if there is a varchar field it drops to 130 rows/sec.

I tried to connect by QLik, Pentaho and Excel and I always have a best performance than Talend, but I need to use Talend due to others integrations. 

I Already updated the JDBC driver, I don't know if I have to install it directly in Talend but in the server, I have the last version of the driver.

The server is in the same network of Database.

We are planning to buy an enterprise version from Talend but I must to prove the efficiency of it.

Thank You very much!

[vapukov · Accepted solution]

Hi again! 🙂

I'm afraid - you are trying to find an answer not on proper resource!

"not each Talend problem related to Talend" (c)

from Informix point of view - Talend it is just a JDBC client and your idea about conversion to some internal format is not correct

First of all - and excel and Qlik - uses ODBC to connect to Informix

I think you need to search on Informix + Java related resources

for example:

https://informixdba.wordpress.com/2017/04/03/improving-remote-query-performance-by-tuning-fet_buf_si...

https://www.ibm.com/developerworks/community/blogs/idsteam/entry/fbs?lang=en

http://ict.udlap.mx/people/roberto/jdbcis341/docsAndothers/doc/release/trouble.fm2.html

all 3 interesting, read them

your better result with Pentaho could be because - different jdbc driver, or different default settings for this driver

but I sure here not too many experts in Informix performance, it is more DBA skills rather than Talend

regards and good luck with resolving the issue! Vlad

[Anonymous]

Thank you Vlad,

You are right some Informix problem here... it is hard to find some solution, but your links are very useful!