topic Re: Talend tFlowMeter returns no result in amc database in Talend Studio

Talend tFlowMeter returns no result in amc database

WSyahirah21 — Fri, 15 Nov 2024 23:01:19 GMT

I am trying to pass the logs, stats and flowmeter data from my spark job to amc database in Talend. Currently, I managed to push the logs and stats data into db (by setting up the amc connection in Stats and Logs in Job tab of tSystem component.). Meanwhile, for flowmeter, I am not sure how to pass that data into amc database.

My effort is:

Checked on the Use volumetrics(tFlowMeterCatcher).

Manually added the tFlowMeter and tFlowMeterCatcher component in talend workflow.

Reference : https://www.talendforge.org/forum/viewtopic.php?id=18319

However, none of these steps working and they still return an empty result in flowmeter table in amc database. Did I miss any steps to fetch the flowmeter to amc database?

Please advice.

Re: Talend tFlowMeter returns no result in amc database

Anonymous — Wed, 13 Apr 2022 07:30:53 GMT

Hello,

As the tFlowMeterCatcher component catches the processing volumetric from the tFlowMeter component and passes them on to the output component, actually tFlowMeterCatcher will count the no. of records pass by the specify flow.

From your screenshot, I don't see any data flow passed in your row1 and row6, row2 and row 4. Since no row type that can be captured by tFlowMeterCatcher, so it is normal that the flow table is empty.

Best regards

Sabrina

Re: Talend tFlowMeter returns no result in amc database

WSyahirah21 — Sat, 16 Apr 2022 07:44:16 GMT

Greetings Sabrina,

This is the full snippet of the workflow connected to the tFlowMeter component.

The workflow is like :

tJava capture the data from its previous component using globalMap.put("execution_time",executionTime.trim()); .
tIterateToFlow will pass the data to tMap and tConvertType. using (String)globalMap.get("execution_time");

which I believe Talend should be able to recognize the record count of this workflow by above components, as the data is passing throughout the workflow. Am I got the right idea of how the tFlowMeter works?