[resolved] Optimize look-up in Talend

[Anonymous]

Hi All,
I need to lookup a table (using tMap) in one of my Talend jobs. However, this table need to be looked up three times based on different join conditions and has around 800k rows.
Please suggest any ways to optimize the performance of the Job given the constraints.
Thanks in advance.

[Anonymous · Accepted solution]

Take a look here ( https://help.talend.com/search/all?query=tMap+lookup+models). You should use the "Load Once" option. This will load your lookup data into memory at the beginning of the job which makes the lookup significantly faster than loading it 800k times. You can make this quicker by loading the data into a tHashOutput component in a subjob before the subjob with the lookup and then use tHashInputs (connected to the tHashOutput above) as many times as you need them. This will mean that only one query will be fired against your database and the data will be reused for as many lookups as you need.