Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us to spark ideas for how to put the latest capabilities into action. Register here!
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Talend performance decreased after migration of DB from postgre to amazon aurora

The problem is related to the Talend performance.

The DB was earlier on PostgreSQL and in Talend, I am using the component tDBInput(PostgreSQL) and the execution time was 10 minutes.

Now the DB is migrated from PostgreSQL to Amazon Aurora Postgre and in Talend, I am still using the same component because of:

1. Amazon Aurora is compatible with PostgreSQL.

2. AmazonAuroraInput component of Talend supports MYSQL compatibility only.

The job execution time is now increased to 30-35 minutes.

Is anyone getting the same latency or any idea about this delay?

Labels (2)
3 Replies
Anonymous
Not applicable
Author

Where is your Talend job running and where was your PostgreSQL DB before you migrated to Amazon Aurora Postgre? My assumption is you've gone from on-premises (Talend) working with an on-premises (PostgreSQL) to on-premises (Talend) working with Amazon Aurora Postgre. If this is the case you have to be aware that Talend is having to retrieve your data over a network that is almost certainly slower than your internal network, then it is having to send the data back (assuming you are sending the data back to your AWS environment). I'm assuming that you are using Talend Open Studio for this. What you could try to improve performance is to compile the job and run it on an AWS EC2 machine.

Anonymous
Not applicable
Author

Hi Rhall,

 

Thanks for the response.

 

My Talend job is at location us-west-2. For DB, there is no change in location as well. PostgreSQL and Amazon Aurora Postgre both are in us-west too.

PostgreSQL was also on the AWS side.
My job is already running on an AWS EC2 machine.

Anonymous
Not applicable
Author

I see. Sorry, I had to go with my first suspicion as it was most obvious 🙂

I can't think why this would be the case, but unfortunately I do not use either of these DBs so cannot really test this out myself. Have you been able to test the queries fired by Talend using another tool? That is the first thing I would do. If there is a significant difference in performance then you know where the issue is (ie with Talend and not with your Aurora  instance). If there is no significant difference it might be an idea to use the Amazon Performance Insights.

Sorry I couldn't be of more help. Maybe somebody else on the Community has seen this scenario and can offer some better advice.