topic reuse look up flow data in Talend Studio

reuse look up flow data

_AnonymousUser — Sat, 16 Nov 2024 12:53:37 GMT

it will be faster if a look up data is reused when it's lodaed twice or more in a job.

Re: reuse look up flow data

Anonymous — Wed, 01 Jun 2011 03:34:34 GMT

Hi
You can store lookup data into memory using tHashOutput component, and then reuse the lookup from memory, it will be faster than loading the lookup each time from input source. For example:
tMysqlInput--main--tHashOutput
|
onsubjobok
|
tFileInputDelimain---main--tMap---tLogRow
|
lookup
|
tHashInput(link to the tHashOutput)
Load model: Reload at each row.
Best regards
Shong

Re: reuse look up flow data

Anonymous — Thu, 02 Jun 2011 18:43:20 GMT

Hi Shong
I have a doubt here. Isnt the reload at each row goes slow as number of record increases in main file.
I mean, If the main file has 1 milion records and look 1 milion.
Please correct me if am wrong