topic Re: Pushing Down in Talend Studio

Pushing Down

Anonymous — Wed, 21 Mar 2012 15:59:55 GMT

Hi,
I am joining two tables from Oracle one is a lookup table with one row and another table is the main with 17k of records.
I am joining both tables in a TMap and outputting to another DB Table on SQLServer.
When running the job it brings all 17k of records into memory rather than pushing down the join to Oracle and bringing back one record.
Is there an easy way of getting joins to pushdown to the DB without using views or TOracleInput and placing the join in there? (which seems to be the only way)
Thanks
Spanky

Re: Pushing Down

Anonymous — Thu, 22 Mar 2012 03:08:11 GMT

Hi Spanky
Welcome to talend community!
I'm not sure whether i understand you correctly.
Here is another workaround.
tOracleRow-->tParseRecordSet-->tMSSQLOutput.
Edit sql to join these two tables in tOracleRow.
But I don't think it will optimize the performance of this job.
If you don't want to cost so much memory, tmap can store temp data in disk.
Regards,
Pedro

Re: Pushing Down

Anonymous — Thu, 22 Mar 2012 07:34:26 GMT

Hi Pedro,
Thanks for the reply, my key aim is reduce bringing back more records than I need into Talend by getting the SQL to join on the database not Talend.
Thanks
Spanky

Re: Pushing Down

alevy — Tue, 03 Jul 2012 02:51:02 GMT

Perhaps make the 17k table the lookup and then in tMap change the lookup model to "Reload at each row". The doco explains how to then ensure the lookup reads only the row you want.