Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik GA: Multivariate Time Series in Qlik Predict: Get Details
cancel
Showing results for 
Search instead for 
Did you mean: 
dwhdeveloper
Contributor
Contributor

Updating few rows in a very large table

Hello, 
I have created a delta load using Talend. 
Everything is working fine except updating the columns seems to take forever. 
In tPostresqlOutput I am using 'Update' in Action on data. 
The table to be updated has more than 7 million rows but I am updating only 500 rows but still it is taking too much time. 
Is there a way to fix this issue? 
I might be updating even bigger tables later so this is a big problem for me.
Labels (2)
5 Replies
TRF
Champion II
Champion II

Hi,
What if you do the same outside from Talend?
dwhdeveloper
Contributor
Contributor
Author

Data is coming from mysql and going to postgresql database with Talend. 
I tested by updating around 1000 rows with Pgadmin it took 3 seconds to update. 
TRF
Champion II
Champion II

Can you share your job and tPostresqlOutput  component config?
vapukov
Master II
Master II

what about indexes?
what column You use for update key? and what real keys in table?
dwhdeveloper
Contributor
Contributor
Author

I used id for update key it is an integer. 
I found a way to speed it up. I inserted updated rows in a temp table and updated the target table from this temp table. 
This really made it fast.