topic Re: tDenormalize issues with to many rows in Talend Studio

tDenormalize issues with to many rows

MRoss1652309981 — Fri, 15 Nov 2024 22:55:45 GMT

Running into an issue I did not see while using 7.2, when I get more than 40 rows back from the database tDenormalize returns blank, if under 40 it works as expected and combines the rows seperated by a comma. Thoughts?

Re: tDenormalize issues with to many rows

Anonymous — Thu, 12 May 2022 02:46:52 GMT

I made a simple testing with v8.0.1 and I can't reproduce the issue, I use a tRowGenerator to generate 50 rows for testing.

Is it a job migrated from v7.2? Add a tLogRow between tMap and tDenormalize to print the data on the console, make sure there has data passed to tDenormalize.

From your screenshot, I see the status is 'starting...' , it seems there is a connectivity issue on the first component tDBInput..., the component is still running all the time?

Regards

Shong

Re: tDenormalize issues with to many rows

MRoss1652309981 — Thu, 12 May 2022 15:37:08 GMT

The size of the rows were fairly big which I think is the problem, seems to have a data cap of some kind.

row example below, my goal was to have 100 of these separated by comma to be sent via an api call.

{ "module": "c8eb72fd-a502-41b6-b96f-4926f92f500a", "id":"c8eb72fd-a502-41b6-b96f-4926f92f500a", "action": "c8eb72fd-a502-41b6-b96f-4926f92f500a", "data": {"id":"c8eb72fd-a502-41b6-b96f-4926f92f500a","agris_id_c":"430-072552-01","ds_settle_num_c":"002420","name":"430-072552-01","ds_net_bu_applied_c":"15000.0","ds_status_paid_c":"P","ds_date_deliv_sheet_c":"2020-07-14","ds_net_bu_customer_c":"15000.0","ds_number_c":"072552","ds_purch_sale_c":"P","ds_settle_loc_c":"430","settlement_agris_id_c":"P-430-002420-","ds_deduct_dollars_c":"93.75","PLACEHOLDER_remaining_dollar_discount":"93.75","ds_deduct_bushels_c":"0.0","PLACEHOLDER_total_application_unit_discount":"0.0","contract_agris_id":"430-P004889401","contract_short_agris_id_c":"P004889401","ds_commodity_code_c":"10","ds_advanced_amt_c":"0.0","ds_advanced_bu_c":"0.0","ds_apply_num_c":"01","ds_apply_num_reopen_c":"01","ds_contract_loc_c":"430","ds_contract_num_c":"0048894","ds_contract_price_c":"3.3","ds_contract_sched_c":"01","ds_cust_name_id_c_c":"H10621","ds_date_application_c":"2020-07-12","ds_date_avg_ticket_c":"2020-07-11","ds_date_pricing_c":"2020-07-08","ds_date_settlement_c":"2020-07-14","ds_gross_bushels_c":"","ds_in_out_c":"I","ds_location_c":"430","ds_non_direct_haul_bu_c":"15000.0","ds_reopen_status_c":"U","ds_status_estimate_c":"F","ds_status_priced_c":"P","ds_storage_table_c":"0","ds_type_c":"P","agris_createdby_c":"AJA","agris_createddate_c":"2020-07-14 11:04:00.0","agris_modifiedby_c":"AJA","agris_modifieddate_c":"2020-07-14 11:04:00.0","agris_sqlid_c":"80AA6D7C-9EF7-4210-8784-ABF80108D2DE","agris_tenantid_c":"05EE048D-413A-4A60-B962-AA9D017E0248","bean":"DSS01_DeliverySheets_Staging","operation":"insert"}}

any other suggestions on merging the 100 row batches separated by a coma would also be welcome, thanks!

Re: tDenormalize issues with to many rows

Anonymous — Fri, 13 May 2022 08:55:46 GMT

You can also hard code on tJavaFlex to concatenate all the rows, for example:

in begin part:

String result="";

in the main part:

if(result.equals("")){

result=row1.columnName;

}else{

result=result+","+row1.columnName;

in the end part, store the value to global variable or context variable for used later.

context.result=result;

Re: tDenormalize issues with to many rows

MRoss1652309981 — Fri, 13 May 2022 15:26:47 GMT

Thanks for all the help everyone, I was able to get around it with the xmlmap function