Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
The problem is related to the Talend performance.
The DB was earlier on PostgreSQL and in Talend, I am using the component tDBInput(PostgreSQL) and the execution time was 10 minutes.
Now the DB is migrated from PostgreSQL to Amazon Aurora Postgre and in Talend, I am still using the same component because of:
1. Amazon Aurora is compatible with PostgreSQL.
2. AmazonAuroraInput component of Talend supports MYSQL compatibility only.
The job execution time is now increased to 30-35 minutes.
Is anyone getting the same latency or any idea about this delay?
Where is your Talend job running and where was your PostgreSQL DB before you migrated to Amazon Aurora Postgre? My assumption is you've gone from on-premises (Talend) working with an on-premises (PostgreSQL) to on-premises (Talend) working with Amazon Aurora Postgre. If this is the case you have to be aware that Talend is having to retrieve your data over a network that is almost certainly slower than your internal network, then it is having to send the data back (assuming you are sending the data back to your AWS environment). I'm assuming that you are using Talend Open Studio for this. What you could try to improve performance is to compile the job and run it on an AWS EC2 machine.
Hi Rhall,
Thanks for the response.
My Talend job is at location us-west-2. For DB, there is no change in location as well. PostgreSQL and Amazon Aurora Postgre both are in us-west too.
PostgreSQL was also on the AWS side.
My job is already running on an AWS EC2 machine.
I see. Sorry, I had to go with my first suspicion as it was most obvious 🙂
I can't think why this would be the case, but unfortunately I do not use either of these DBs so cannot really test this out myself. Have you been able to test the queries fired by Talend using another tool? That is the first thing I would do. If there is a significant difference in performance then you know where the issue is (ie with Talend and not with your Aurora instance). If there is no significant difference it might be an idea to use the Amazon Performance Insights.
Sorry I couldn't be of more help. Maybe somebody else on the Community has seen this scenario and can offer some better advice.