topic How to update previous row after comparing with current and previous row? in Talend Studio

How to update previous row after comparing with current and previous row?

SSingh1687332860 — Fri, 15 Nov 2024 21:38:37 GMT

Hi, i need to update previous row values after comparing with current and previous row. I am using tMemorizeRows for saving the previous row and tJavaFlex for comparing the previous row with the current row but i was not able to update the previous row with new values. How can i do it?

Re: How to update previous row after comparing with current and previous row?

anselmopeixoto — Tue, 27 Jun 2023 03:29:23 GMT

Hi @Sunny Singh

tMemorizeRows only stores a photography of previous rows, so any changes you make on its array won't affect the actual rows as they already went through the component.

I'd suggest you try using tHashOutput after tJavaFlex, set a key on your schema if not already and set tHashOutput to "Keep last" on key management. This way you'll will be able to output previous row again and simulate a update behavior by keeping only last row based on its key.

Then you use tHashInput on a second subjob to read wha'ts stored in memory.

Re: How to update previous row after comparing with current and previous row?

SSingh1687332860 — Tue, 27 Jun 2023 07:41:00 GMT

Thank you for your answer @Anselmo Peixoto . There is a way to select the row to outoput (current o previous) based on a condition in tHashOutput o tJavaFlex?

Example:

if(condition)

output current row

else

output previous row

Re: How to update previous row after comparing with current and previous row?

anselmopeixoto — Tue, 27 Jun 2023 13:47:26 GMT

Hi @Sunny Singh

Let's say you have a job like this:

input -row1-> tMemorizeRows -row2-> tJavaFlex -row3-> tHashOutput

when current row reaches tJavaFlex, the previous row were already sent to its output in last iteration, so it will be already stored in tHashOutput.

For this reason, what I suggest is that you replace the previous row stored in tHashOutput based on its schema key.

On tJavaFlex main code you can write a condition like in your example, but in those cases you want to output previous row, you actually replace current row key with the same key as previous row.

if(condition) //add a new row to tHashOutput

row3.key_column = row2.key_column;

row3.another_column = row2.another_column;

else //replace previous row

row3.key_column = key_column_tMemorizeRows[1]; //last row key

row3.another_column = row2.another_column; //you can update values for other columns in previous row if you want

Now considering that previous row is already in tHashOutput, if you don't want to update any column for that row, you can just use the "continue;" keyword in your condition. This will make the loop "jump" to next iteration without the need of a output row.

Re: How to update previous row after comparing with current and previous row?

SSingh1687332860 — Thu, 29 Jun 2023 12:39:04 GMT

Hi @anselmopeixoto (Customer)

Thank you for your answer.

i tried all the steps you suggested but it didn't work. Maybe i made some mistakes.

Job: input -row1-> tMemorizeRows -row2-> tJavaFlex -row3-> tHashOutput

This is the code in tJavaFlex:

Start code:

int cont = 0;

Date date_start = new Date();

Date date_end = new Date();

SimpleDateFormat format = new SimpleDateFormat("dd-MM-yyyy");

Main code:

if(row1.A == last_a && row1.Year == last_year && row2.C.equals(last_c) && row1.D.equals(last_d) && row1.E.equals(last_e)) {

if(!row1.F && cont == 0) {

//do nothing

}

else if(row1.F) {

date_end = TalendDate.addDate(format.parse("31-12-"+row1.Year), row1.Months*-1, "MM");

//PreviousRow

date_start = TalendDate.addDate(date_end, Months_tMemorizeRows_1[1]*-1, "MM");

date_start = TalendDate.addDate(date_start, 1, "dd");

row2.KeyColumn = KeyColumn_tMemorizeRows_1[1];

row2.DateStart = date_start; //-> update DateStart in the previous row

row2.DateEnd = date_end; //-> update DateEnd in the previous row

cont = 0;

}

else if(!row1.F) {

//PreviousRow

date_start = format.parse("01-01-"+ Year_tMemorizeRows_1[1]);

date_end = TalendDate.addDate(date_start, Months_tMemorizeRows_1[1], "MM");

date_end = TalendDate.addDate(date_end, -1, "dd");

row2.KeyColumn = KeyColumn_tMemorizeRows_1[1];

row2.DateStart = date_start; //-> update DateStart in the previous row

row2.DateEnd = date_end; //-> update DateEnd in the previous row

//CurrentRow

date_end = TalendDate.addDate(date_end, 1, "dd");

date_end = TalendDate.addDate(date_start, row1.Months, "MM");

date_end = TalendDate.addDate(date_end, -1, "dd");

row2.KeyColumn = row1.KeyColumn;

row2.DateStart = date_start; //-> update DateStart in the current row

row2.DateEnd = date_end; //-> update DateEnd in the current row

cont = 0;

}

cont++;

last_a = row1.A;

last_year = row1.Year;

last_c = row1.C;

last_d = row1.D;

last_e = row1.E

------------------------------------------------------------- End Code ----------------------------------------------------------------------------

What can i do to fix the problem?

Re: How to update previous row after comparing with current and previous row?

anselmopeixoto — Thu, 29 Jun 2023 19:14:05 GMT

Hi @Sunny Singh

Unfortunately I cannot test your code, but what caught my attention is that I didn't find where the last_a and other last* variables are being first declared/initialized.

Perhaps you can find out what is happening by replacing temporally the tHashMap by a tLogRow and limiting the input data to a sample, so you can investigate its behavior.