[resolved] tMap needs to support multiple lookups to the same table (alias)
In a Data Warehouse Star schema scenario it is a common requirement to be able to lookup multiple surrogate keys based on the same physical lookup table. Example: A (main) order record may contain the following columns: OrderDate, ProcessedDate, ShippedDate, etc - all needing separate surrogates from a date dimension in the data pipeline. Therefore the Date Dimension needs to be used as a lookup multiple times. In OLAP terms this is called a role-playing dimension. In database terms this is just an alias.
So, rather than duplicating the Date dimension table multiple times as mutiple tInput lookup components, and using up memory, it would be great to use a single tInput lookup component, and in the tMap, be able to alias that single input lookup multiple times, joined from the desired main input. That makes for a simpler design, and should save valuable memory.
Dave
tHashOuput and tHashInput components exist but they are not official, they don't take account StoreOnDisk mode, and they don't work well when they are directly connected to a tMap.
Yet, I will speak about this subject at our next meeting.
tHashOuput and tHashInput components exist but they are not official, they don't take account StoreOnDisk mode, and they don't work well when they are directly connected to a tMap.
Yet, I will speak about this subject at our next meeting.
tHashOuput and tHashInput components exist but they are not official, they don't take account StoreOnDisk mode, and they don't work well when they are directly connected to a tMap.
Yet, I will speak about this subject at our next meeting.
That would be great. It is extremely common to have a date dimension aliased many times for lookups.
Is there any solution for this? I have to load 3 times the date and hour tables !!!
Im tMap, you can create a lookup based on the same table more than once, but, essentially, you are incurring the extra memory for each duplicated lookup table, nit to mention cluttering up the job display . Also, if the solution adds inferred dimension members on the fly, and that dimension is on eof the dups, then I would doubt that the others would get get updated. - not sure if Talend dynamically updates lookup cache as records are added.
I am used to SSIS (I like Talend better), where the same lookup table can be aliased several times. Also, added inferred dimension members are dynamically added to the lookup cache as well.
Hopefully talend will jump on this one.
Dave
