Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Slow Insertion in Amazon Redshift

Hi,
We have just created a simple job to fetch data from MySQL table (Local database and from Amazon RDS), having rows 300,000 and to insert these rows into Redshift. It took us more than 4 hours to do that.
1. Why is it very slow to fetch data from one single table and to insert it in Amazon Redshift using Talend OpenStudio Big data?
2. Is there a way to do a fast insertion? where it should insert it in less than 5 minutes?
Please find the attached screenshots for details.
thanks!
0683p000009MDkY.png 0683p000009MEOU.png
Labels (4)
24 Replies
Anonymous
Not applicable
Author

Hi naveed_aq,
What does your job design look like? Could you please upload your job design screenshots into forum?
Best regards
Sabrina
Anonymous
Not applicable
Author

Hi naveed_aq,
What does your job design look like? Could you please upload your job design screenshots into forum?
Best regards
Sabrina

i have attached the snapshot, tmap does not have anything fancy, it is just one to one column mapping... 
i am running Talend on an EC2 machine having r3.8xlarge type 2 compute node structure, but insertion speed is just aroud a few hundreds, mostly around 300
Regards.
0683p000009MDkZ.png
_AnonymousUser
Specialist III
Specialist III

Hi,
Did anyone get any luck to increase performance to load data in redshift.
regards
Shankar
Anonymous
Not applicable
Author

Has anyone figured out the fastest way to move data from MSSQL(or MySQL) to redshift.

 

Thanks,

Alok

Anonymous
Not applicable
Author

Hello @AlokGarg 

 

This case is years old and this performance concern on 5.3 was improved greatly however there was no wok around but to upgrade to the latest release. I would recommend attaching your configuration for the commit every, the batch size, the version and other information because it is not possible this is the same issue as before even if the performance is not great for you.