topic tMap lookupModel in Talend Studio

tMap lookupModel

Anonymous — Tue, 27 Mar 2018 15:34:38 GMT

Hi Team

In tMap we have Loop models in which we find reload at each row option which loads the reacords for evey input now based on the source column lookup data.

Is this a optimized technique?, for every run data will be loaded to chache.

Instead if we go for the normal procedure the entire data is loaded once where hits to db is reduced

Can u suggest is reload at each row is optimal?

Thanks

Deepthi

Re: tMap lookupModel

Anonymous — Tue, 27 Mar 2018 16:29:00 GMT

Hi Deepthi,

Its depends on the situation, size of the table and refresh rate of the main and look up table.

if the lookup is relatively small its better to use Load Once method.

There are 2 instances I can think of using reload at each row.

-If the lookup table refreshes quite often if you have long running job

-If you process minimal record at a time don't need to bring a big lookup table into the memory

I hope this helps.

Cheers!

Gatha

Re: tMap lookupModel

Anonymous — Wed, 28 Mar 2018 04:35:32 GMT

Thanks for your comments

But dont you think the i/o operations to the db will be more

if 100 records are coming from source for each record data is pulled from the lookup ryt

Thanks

Deepthi

Re: tMap lookupModel

Anonymous — Wed, 28 Mar 2018 10:33:20 GMT

Hi,

Think about a situation where your lookup table is bigger than the Main table,

Like your example 100 records from main stream, millions of rows in lookup table in that case 100 small i/o operation is better than one huge extract which is going to take more time.

I hope this helps.

If you try the above example, you will have a better idea.

Cheers!

Gatha