Slowness in Data load from Talend to AWS Redshift.
Hello,
Is anyone facing issues with slowness in data load via talend studio or TMC since yesterday morning 5 am CET. I checked with my IT Team and they rebooted all the possible servers, but still the issue persists. our target DB is Amazon Redhsift DB.
the output for some of the SCD1/SCD2 jobs is like 2500 rows in 2.5 hrs. usually it used to take just 30 t0 40 mins to process this much data. anyone facing the same issue please let me know OR can suggest a quick resolution as our prod DW loads are stuck up.
@achen: we use AWS redshift as our database and do not have control to modify the redshift configurations i think so since we have the cloud data based. So are talking about making changes to the cloud database or the on premise database?
@Aiming Chen : as per your suggestion, i tried reducing the cursor size to 1000 and running the job again. it took 32 mins to process almost 700 rows with postgresql component( SCD1/SCD2 logic). we already have a sort key defined on this table in redhsift database. so i think its not making any diference with this also.
@Aiming Chen : finally the db changes worked which were done by my IT team. not sure exactly what did they do though. thanks for your suggestion. the issue is now resolved 🙂
