How to delete rows based on aggregate condition

dattasud — Sat, 16 Nov 2024 02:59:30 GMT

Hi All,

In my use case, I have to remove rows from my data set based on an aggregate condition (if the aggregate count is > 20 then remove those items from the data set). Below is my job flow.

From the tMap_1 I have two same output. Main order: 1 - will then passed to taggregaterow_1 on one column and tfilterrow_1 will return the column with count >20.

On the Main order:2 - I have a unique row check criteria. After that I want to take out all the filteredrow_1 results(row4) from the row5 results.

I tried to achieve this by storing data in tHashoutputs and getting it back in tHashinputs and used tMap_2 with left outer join to take only main rows without the rows having count >20. With this I am getting below error:

Exception in component tHashInput_2 
java.lang.RuntimeException: The hash is not initialized : The hash must exist before you read from it

I have three question from here:

1. How can I execute Main order:1 first then Main order:2 of tMap_1?

2. Can I do a wait to initialize tHashInput_3 and tHashInput_2 untill corresponding tHashOutput_1 & _2 has the values ?

3. Is my approach correct to solve the problem?

Please note, I checked the tAggregateRow/tFilterRow results, with the condition it will have 26(count) results.

Please help!

Re: How to delete rows based on aggregate condition

dattasud — Fri, 13 Mar 2020 15:05:40 GMT

Exception in component tHashInput_2 
java.lang.RuntimeException: The hash is not initialized : The hash must exist before you read from it

Above error, I am able to resolve by removing the 'OnComponentOK ' to tHashInput_3 from tDBInput as it was on priority to execute.

Re: How to delete rows based on aggregate condition

dattasud — Fri, 13 Mar 2020 15:34:30 GMT

ALso, I used tJoin in place of tMap for inner join reject condition to take out tAggregateRow/tFilterRow results from main data set.

topic Re: How to delete rows based on aggregate condition in Talend Studio

How to delete rows based on aggregate condition

Re: How to delete rows based on aggregate condition

Re: How to delete rows based on aggregate condition