topic Re: How to test a value on a previous line? in Talend Studio

How to test a value on a previous line?

amounier — Sat, 16 Nov 2024 13:32:39 GMT

I am using TOS to add a colum to a flat file.
Working with
Talend Open Studio Version: 4.0.0M2 - Build id: r35653-20100114-0503
OS: Microsoft Windows Server 2003 Stabdad x64 Edition Service Pack 2
Java
The input file has several colums, and the output file should have the same colums in a different places and in addition one more colums called DNOR. For my exemple, I only show 1 colum and the DNOR one.
The added column called DNOR in my exemple has a type integer.
The DNOR value starts at 1 for the first line and should be increased by 1 each time the value of the colums called ACCOUNT is different from the previous line.
Therefore the rules is as follows:
If the ACCOUNT_Current_Line!=ACCOUNT_Previous_Line then DNOR++

INPUT file description
ACCOUNT;
641100;
641100;
641100;
421000;
421000;
841999;
841999;
841999;
841999;

OUTPUT file description exemple
ACCOUNT;DNOR;
641100;1;
641100;1;
641100;1;
421000;2;
421000;2;
841999;3;
841999;3;
841999;3;
841999;3;
I did read the documentation for several inegrated fonction of TOS, and look for the same problematic in the forum but I did not find the clue to implement this. I try using the tJavaRow, the tFilterRow and the tMap, but did not succeed each time when I try to get the ACCOUNT_Previous_Line value.
If someone can give me the entry point on how to handle such request, I would be very gratefull.
Regards
Aymeric

Re: How to test a value on a previous line?

Anonymous — Wed, 17 Feb 2010 10:39:18 GMT

To generate a correlation key, use tjavarow and follow the concept in my post
https://community.talend.com/t5/Design-and-Development/Multiple-Record-Type/td-p/104947
of course it assumes the source is ordered
hope it helps

Re: How to test a value on a previous line?

amounier — Wed, 17 Feb 2010 11:43:46 GMT

Thanks for your replay emaxt6, I look at your post 7663, and I understand the concept however, in my case, I do not have a fixed string to test, in order to increase or not my SYNTHKEY. in place of the following if test:
if(input_row.Col1.equals("G203")) {
//gen new key and store it
mykey++;

I am looking for a test I could do on the previous row for exemple
if(input_row.Col1.equals(input_row.Col1_PREVIOUS_ROW")) {
//gen new key and store it
mykey++;
The trouble is that I do not manage to use the global variable in order to compare the previous row value to the current row value. Any idea on how to emplement this in the tJavaRow or better directly in the tMap?

Re: How to test a value on a previous line?

Anonymous — Wed, 17 Feb 2010 12:30:38 GMT

try this as start:
------
int mykey = (Integer)(globalMap.get("MYKEY") == null ? 0 : globalMap.get("MYKEY"));
String prev = (String)(globalMap.get("PREV") == null ? "< __NULL_!!__>" : globalMap.get("PREV"));
if(!input_row.MYFIELD.equals(prev)) {
//gen new key and store it
mykey++;
prev = input_row.MYFIELD;
globalMap.put("MYKEY", mykey);
globalMap.put("PREV", prev);
}
output_row.MYFIELD = input_row.MYFIELD;
output_row.MYKEY = mykey;
-----

Re: How to test a value on a previous line?

amounier — Wed, 17 Feb 2010 14:33:01 GMT

Dear emaxt6,
Thanks so much, this is working very well and I manage to adapt a little your code, this produice exactly what I wanted.
Thanks again and thnaks for your time and effort.
Regards
Aymeric

Re: How to test a value on a previous line?

Anonymous — Sun, 21 Feb 2010 14:22:11 GMT

You are welcome.
Anyway I think that would be very useful if Talend would consider to provide a component ie. named tPreviousRecords, that a user can put inline in a flow from the gui and such component will "memorize" up to n previuos records (default one record) that a user can then simply refer in other components in the job (ie. tmap) as other component published variables.
Does someone at Talend agree? It would solve a lot of use cases and provide great flexibility.
thanks

Re: How to test a value on a previous line?

Anonymous — Tue, 13 Apr 2010 12:54:46 GMT

Feature requested and handled by Talend very fast
http://www.talendforge.org/bugs/view.php?id=11720
thanks