Catch processed row's count

[Anonymous]

Hi there,

I'm using TDI open studio. I have designed and run the job, whose screenshot is :

As you can see, there we are getting count of processed rows in green colour. I want to catch these count in separate file (prefer excel file).

So for above job, I expect below output :

After researching for a while, I come to know that there are 3 components that I feel one of these 3 can fulfill the requirement.

tStatCatcher

tLogCatcher

tFlowMeterCatcher

But I don't know which one to use and how to use it for my requirement. Please, help me out guys. 

Thank you

[Anonymous · Accepted solution]

Hello,

The tFlowMeterCatcher component catches the processing volumetric from the tFlowMeter component and passes them onto the output component.

For example:
tFileInput--->tFlowMeter--->tMysqlOutput_1
tFlowMeterCatcher---tLogRow
There is one column called "count" in the schema of tFlowMeterCatcher which counts the no. of records pass by the specify flow.
Let me know if it is OK with you.

Best regards

Sabrina

[Anonymous]

Hi @xdshi

Thank you so much. Your solution worked !

I tried 2 options :

1. Suggested by you :  using  tFlowMeterCatcher.- easy to record statistics in csv or excel file as it has output row.

2. Without using tFlowMeterCatcher - tried stats&log under job setting tab, where we check the box in front of 'Use volumetrics' and specify the file path and name. When we use this, we don't need to use tFlowMeterCatcher component. But its giving output file with values separated by ' ; ' (semicolon).

How can we get the the output that can be view properly as csv or excel using 2nd option ?

(might be comma separated, or using encoding, etc.)

Thanks

[Anonymous]

Hello,

We see that you have saved your 'Meter file name' as meter_file.csv

Could you please show us your actual output result with values separated by ' ; ' (semicolon)?

Please mask your sensitive data.

Best regards

Sabrina

[Anonymous]

Please find below the screenshots of the output result :

opened in ms excel :

opened in notepad :

[Anonymous]

Hello,

It should be field separator to separate fields for the transferred data when set the flow meter log to output the count record into a file.

Best regards

Sabrina

[Anonymous]

Where is the option to set field separator ?
I can see only encoding option there in job setting-> stats&log -> use volumetrics -> on files.
is there anything by which we can get comma instead of semicolon ?

[Anonymous]

Hello,

We will make an investigation on it and then come back to you as soon as we can.

Best regards

Sabrina

[Anonymous]

Hi,

In tFlowMeterCatcher, we are not getting the duration of processing.
I come to know that there is component called tChronometerStop to get a duration. But its logging the time on console. How can I get this time in file ? Or is there workaround for the same ?

[Anonymous]

---

@AkshayNG wrote:
Hi,

In tFlowMeterCatcher, we are not getting the duration of processing.
I come to know that there is component called tFlowMeterCatcher to get a duration. But its logging the time on console. How can I get this time in file ? Or is there workaround for the same ?

---

If I understand your issue clearly, do you want to use component tStatCatcher in your work flow and log the output into file? If so, you can design your work flow like: tStatCatcher-->tFileOutPutxxx(your target output file).

Best regards

Sabrina