unique count of rows in a file

[Aish123]

Hi 

i have a text file which has 2 key elements, depending upon on the key elements i have to fetch the unique row count and store it in a file.

Please let me know how this can be acheived.

Thank you.

[billimmer]

use tUniqRow to get the count.

[Aish123]

Hi,

Thanks for the reply.

 I tried with the tuniqrow component.

Tfileinputpostional-->tuniqrow-->tlogrow

In this, i am getting the unique rows data on the console..i need the unique count and want to store the same in an file.

I also tried with tfileinputpositional->tuniqrow->tflowmeter->tflowmetercatcher-->tfileoutputdelimited.

In this, there are 0 rows processed at the tfileoutputdelimited and there is no data present in it.

[Anonymous]

You don't use tFlowMeterCatcher component correctly.
There is a pre-defined global variable on tFileOutputDelimited that counts the total number of rows are output into file, you can use this variable on other subjob.
((Integer)globalMap.get("tFileOutputDelimited_1_NB_LINE"))
eg:
tfileinputpositional->tuniqrow-->tfileoutputdelimited.
|onsubjobok
tJava

on tJava:
System.out.println("the number of unique rows is "+(Integer)globalMap.get("tFileOutputDelimited_1_NB_LINE"))

Regards
Shong

[Aish123]

Hi Shong,

I tried the approach suggested by you, but it is throwing an error.

Please find the attached screen print for the same.

Thank you.

error.PNG
job1.PNG

[Anonymous]

It is a compilation error, maybe you need to synchronize the schema after you modify the job, check the schema and basic settings of tFileOutputDelimited.

BTW, please insert images into your post body instead of attaching them!

[Aish123]

Hi Shong,

I am getting an unique data as an output in the delimited file instead i want to store the count of unique rows in the file.

I want to store the count "2" as seen on the console in the file.

[Anonymous · Accepted solution]

OK, just use a tJavaFlex instead of tFileOutputDelilimited, define a counter on tJavaFlex, eg:
begin part:
int counter=0;
main part:
counter++;
end part:
globalMap.put("counter",counter);

In next subjob, get the value of this global variable and output it to a file.
|onsubjobok
tFixedFlowInput--main--tFileOutputDelimited

on tFixedFlowInput, add one column with int type, and set its value as:
(Integer)globalMap.get("counter")

[billimmer]

You don't need all that.  The tUniqRow already returns:

NB_UNIQUES: the number of unique rows. This is an After variable and it returns an integer.

NB_DUPLICATES: the number of duplicate rows. This is an After variable and it returns an integer.

You can write out the value of NB_UNIQUES to a file or DB table with tFixedFlowInput.

[Aish123]

Hi Shong,

Thanks it worked successfully.