topic Re: How to get the rowcount when an exception occurs in Talend Studio

How to get the rowcount when an exception occurs

Anonymous — Sat, 16 Nov 2024 10:34:09 GMT

Hi,
I have a requirement where i have to get rowcount even when an exception occurs in the middle of the job
I have used NB_Line global variable to get no of records from tMongodbinput component,but it is working fine only when no exception occurs.when any exception occurs during the execution of job, records are getting transferred to target but rowcount is showing NULL.
Can anyone tell how to get rowcount when exception occurs during execution of job.
Thanks in Advance,
Prasanthi B.

Re: How to get the rowcount when an exception occurs

Anonymous — Wed, 22 Jun 2016 04:07:09 GMT

Hi,
Usually, we use tFlowMeterCatcher to catch the processing volumetric from the tFlowMeter component and passes them on to the output component.
Flow Meter table can record the number of rows processed in the defined flow(prerequisite: you have to use tFlowMeter in your job).
Please take a look at this component reference with scenarios.
Let us know if it is OK with you.
Best regards
Sabrina

Re: How to get the rowcount when an exception occurs

Anonymous — Wed, 22 Jun 2016 07:51:45 GMT

Hi,
Thanks for the reply.
I tried using with tFlowMeter and tFlowMeterCatcher in the job, but it is working fine when no exception occurs.
when any mongoDB exception occurs in the job records have been processed to target but cant able to get count of records moved to target before exception occurs.
can anyone tell how to get no of records fetched from source and inserted in to target even when any exception occurs.
Please find the attached screenshots of the job.
Thanks in Advance,
Prasanthi B.

Re: How to get the rowcount when an exception occurs

Anonymous — Thu, 23 Jun 2016 04:02:54 GMT

Hi,

can anyone tell how to get no of records fetched from source and inserted in to target even when any exception occurs.

Do you mean the no of records fetched from source and inserted in to target before an exception occurs?(When an exception occurs, the job will stop and there is no processing data)

Usually, we use Nb_LINE is used to count the total number of records have been proceed.

When inserted item is choosen, there will be a global variale such as ((Integer)globalMap.get("t<DB>Output_1_NB_LINE_INSERTED")) will be avaible, which counts the total number of records have been inserted into database.

Best regards

Sabrina

Re: How to get the rowcount when an exception occurs

Anonymous — Thu, 23 Jun 2016 07:12:41 GMT

Hi,
Thanks for the reply.
Yes,Before an exception occurs how many records have been fetched from source and inserted into target in the job.
Rowcount (fetch count and insert count) is getting successfully with NB_Line global variable only when no exception occurs ,but any exception occurs during the execution of the job then unable to get no of records have been fetched from source and inserted into target.
Kindly help me.
Thanks,
Prasanthi B.

Re: How to get the rowcount when an exception occurs

_AnonymousUser — Thu, 23 Jun 2016 07:50:34 GMT

Hi xdshi ,
This is same scenario with me. By the time exception occurs, some records are being transferred. There is no indication of those records that transferred as well as fetched from ((Integer)globalMap.get("t<DB>Output_1_NB_LINE_INSERTED")) and ((Integer)globalMap.get("t<DB>Input_1_NB_LINE")).

Re: How to get the rowcount when an exception occurs

Anonymous — Thu, 23 Jun 2016 08:38:35 GMT

Hi balamPrasanthi,

Would you mind posting your current job design screenshot into forum? Did you try to get the row count in a standalone subjob by using this ((Integer)globalMap.get("t<DB>Output_1_NB_LINE_INSERTED"))?(tfixedflow-->tlogrow to print the no of processing data out console directly)?

Best regards

Sabrina

Re: How to get the rowcount when an exception occurs

Anonymous — Thu, 23 Jun 2016 08:48:48 GMT

Hi chaitanya77,

What's kind of indication do you want to get? Could you please elaborate your case with an example with expected output values?
Best regards
Sabrina

Re: How to get the rowcount when an exception occurs

_AnonymousUser — Thu, 23 Jun 2016 10:13:49 GMT

Hi xdshi,
In my case, I transfer records from source mongoDB and destination MSSql. If I have 100 records in source(mongoDB) and I try to move them to destination, and let 40th record is corrupted record which throws exception. By the time 40th record being processed already 39 records are transferring to destination(MSSql). At 40th record only tMongoInput is throwing exception and job stops, by the time which already 39 records getting transferred.Now when I trying to get the values from ((Integer)globalMap.get("tMongoDBOutput_1_NB_LINE")) it is giving null value.But if I check destination 39 records are present.

Re: How to get the rowcount when an exception occurs

Anonymous — Thu, 23 Jun 2016 10:27:38 GMT

Hi xdshi,
Thanks for the quick reply.
Please find the below screenshots of the job and the console log when exception occurs during execution of job.
We used NB_Line global variable in tjava component.When job ran successfully, records are successfully moved to target and records count is successfully getting by this ((Integer)globalMap.get("tMongoDBInput_1_NB_LINE")) and ((Integer)globalMap.get("tMongoDBOutput_1_NB_LINE")) global varaiables in the job.
But when any mongo exception occurs due to improper data,records are successfully moved to target but how many records fetched from source and inserted into target are not getting by this global variables before the exception occurs.
Can you please tell me the solution in finding the recordscount(how many records fetched from source and inserted into target before exception).

Thanks,
Prasanthi B.

Re: How to get the rowcount when an exception occurs

Anonymous — Thu, 23 Jun 2016 10:32:56 GMT

Please find the below screenshots.