I want to sum up the values from the amount column, for each account (account_code), for each day (payment_date) -> only for the records where transaction_type is RFP or RFX. So in the above data, for 7H0068T1, I need to sum up 2521.4 and 678.6 (and exclude WHT/ 800.0)
The iAggregate node doesn't seem to work for me or I am missing something.
Technically it's a combo of 'group by' and 'where' and the tAggregate only provides me the option to group by
If you are using a DB, why you need to combine the dataset? You can add a tDBOutput to each stream so that each dataset will be loaded independently.
Warm Regards, Nikhil Thampi
Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂
I don't think DB will have any serious impact if we hit 3 times instead of 1 especially in batch mode.
If you want to use a single session, you can use a tDBConnection at the beginning and use the same connection for all 3 DB objects. End of day, you will have to use DB resources to push based on the number of records.
But if you really want to collect the data to one dataset and then push to DB (definitely not my preferred method), you can first collect the data to three tHashOutput components (where the second and third hash will be in append mode to first one). Then read this Hash using a tHashInput and then push to DB (this means memory overhead due to usage of Hash).
You can use files instead of Hash to do the same operation but this means you are doing additional IO and CPU operations. So my view is that the optimum way is to push through three DB components.
Warm Regards, Nikhil Thampi
Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂
