Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in Bucharest on Sept 18th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Data insertion performance using Talend DI Jobs

Talend version - 6.1.1. 
SVN Version - svn, version 1.6.11 (r934486) 
CentOS release 6.7 (Final) 
DB - MySQL 5.7.13 InnoDB Engine
Infrastructure hosted in Azure Cloud
Environment - Pre-PROD

We are facing performance issues while inserting the data into MySQL DB using the Talend jobs deployed in TAC.
Problem Statement :
# Job design - with only insert in tmySQLoutput component, data flow rate is >2000rows/sec.
# Job design - with only update in tmySQLoutput component, data flow rate is >2000rows/sec
# Job design - with insert or update in tmySQLoutput component, data flow rate is very poor. approx 3-4rows/sec.

We are having the same architecture in development environment. in DEV env, even jobs with design as insert or update in tmySQLoutput component, data flow is far better >1000 rows/sec.

Talend DI Server & MySQL DB versions are same in both the environments. But facing the data flow issue in Pre-PROD env.

Can anyone provide help on this issue.

Thanks
Ashok Karnam

Labels (2)
1 Reply
Anonymous
Not applicable
Author

Hi,
What does your job design look like? Have you already checked out "Extend Insert " option on tMySqlOutput component to carry out a bulk insert of a defined set of lines?
 Have you already reported a ticket on Talend Support Portal for your subscription version? In this way, we can give you a remote assistance through support cycle with priority.
Best regards
Sabrina