Can we use Talend generated variables in tmap expression.
Hi,
I am using Talend's collection variable in Tmap expression editor for some logic. Would using these variables in my code be the proper way to use them?
If Talend changes the existing variable standards/conventions in future, then would that affect my existing code (may be while rebuilding)?
Thanks,
Atul
Hi,
Could you please give us more information about your job requirement? Setting an example will be preferred. Can you create a user routine?
Best regards
Sabrina
Hi Sabrina,
My exact requirement is compare each records with previous one. If it matches then update that record with previous record's columns(when there is null/no value) and keep that record in memory for comparison with next record.
I did it using following,
tJDBCInput --> tMemorizeRows --> tmap --> tLogRow
Here I am using tMemorizeRows to store previous record and tmap to compare that record with current record.
So for first record no comparison, record will print as it is. But for next record comparison will be performed and here current record's data will be replaced with old record(only for columns that has null/no values).
This works fine but here is the challenge,
Now i want tMemorizeRows to store the updated record for next record. When next record enters into tmap for comparison, tMemorizeRows will give old state of previous record (i.e. Old record with null values but needed all not null values).
I have achieved this by using one of the variable that Talend creates when we create an output in tMap. This is Talend's internal variable but accessible from tMap expression editor so I have used. Variable name will be Out1_temp if we creates the tMap output as "Out1".
My question is, is it a good practice to use such type of Talend internal variables in tMap, tJava or any such components where we could write expressions or Java code?
Please let me in case more information is required.
Thanks,
Atul
I suggest using a tGlobalMap or using a java component for this use case. I feel that this use case is a weakness of Talend vs. other ETL tools, but I rarely need it so I haven't created an enhancement request. For example, in DataStage the variables in the mapper retain their values rather than reset for each row. I would be interested to hear how others have handled this use case in Talend.
You can use tHashInout and output component for this scenario, make i copy of all the rows in tHashOutput then use same tHashInput along with original flow for comparison you will have better control over all the records.
umeshrakhe's suggestion is great. The last time I needed to do this I was in open source and didn't have hash components available. You may wish to add a line number to your data using a sequence to make comparing to the previous row simpler.
This use case seems similar to another post here: https://www.talendforge.org/forum/viewtopic.php?id=41451
Please see that post for an approach that does this using tAggregateRow and the List function for accumulation. this may be a more robust solution for you since there is no need for internal variables, global variables, or even tHashMap.
