topic Re: [resolved] unable to count file row count in Talend Studio

[resolved] unable to count file row count

Anonymous — Wed, 04 Jan 2017 08:15:46 GMT

Hello,
I have a job which captures all details of the files in a particular folder.Like file_rowcount , size or file_name,
But i am unable to capture file_rowcount
Please see my job designe:-

tfilelist:-

tfileproperties:

tfilerowcount:-

tmap:

the output i am getting correct except count is not matching with the file , its giving any random xyz figure.
Please help here
Thanks.

Re: [resolved] unable to count file row count

Anonymous — Wed, 04 Jan 2017 08:52:03 GMT

Hi
It is a job design issue, the global variable should be used after component, it should be:
tfilelist--iterate--tfilerowcount--oncomponent--tfileproperties--main--tmap.....
Regards
Shong

Re: [resolved] unable to count file row count

Anonymous — Wed, 04 Jan 2017 09:19:20 GMT

Dear Shong,
Thanks for reply,
i just did what you said , same issue , row-count coming incorrect , even if file has only header , its giving count as 44.

All logic is same. just shifter the positions as suggested by you.
but same issue, even file has no records still it shows any random figure.

Re: [resolved] unable to count file row count

Anonymous — Wed, 04 Jan 2017 09:34:07 GMT

Dear Shong,
Its resolved,
row separator for tfilelist was wrong , default should have been "\n".
Thanks.

Re: [resolved] unable to count file row count

Anonymous — Wed, 04 Jan 2017 11:02:41 GMT

Great, thanks for your feedback!

Re: [resolved] unable to count file row count

Anonymous — Wed, 04 Jan 2017 17:21:00 GMT

Dear Shong,
In this job , if there are 10 files in the directory , how will i find the count of files in the same job ?
any global variable function ?
Can you suggest?
Thanks

Re: [resolved] unable to count file row count

Anonymous — Wed, 04 Jan 2017 17:37:27 GMT

Hi ,
You can do it ,it is pretty simple .
tfilelist > tfileinputdelimited>tmap > tfileout >tjava
In tjava component , you can use the global variable ,which will help you to print the filename and row count of file ,which is being processed by tfilelist.Below are the screenshots for the same.

Re: [resolved] unable to count file row count

Anonymous — Wed, 04 Jan 2017 17:50:46 GMT

Dear Harkirat,
Thanks for reply , i want to include the logoc in the same job which i made it , please have a look at it above screen shots are attached

Re: [resolved] unable to count file row count

Anonymous — Wed, 04 Jan 2017 17:53:26 GMT

Hi ,
You can do it ,it is pretty simple .
tfilelist > tfileinputdelimited>tmap > tfileout >tjava
In tjava component , you can use the global variable ,which will help you to print the filename and row count of file ,which is being processed by tfilelist.Below are the screenshots for the same.
195095/mini_blob_20170104-0939.png
195095/mini_blob_20170104-0940.png

i need to find the file count not the file row count

Re: [resolved] unable to count file row count

Anonymous — Fri, 06 Jan 2017 18:03:39 GMT

Dear Shong,
In this job , if there are 10 files in the directory , how will i find the count of files in the same job ?
any global variable function ?
Can you suggest?
Thanks

Dear Shong,
How can i find the total files in a particular folder?
In the same job i want to find the total files , can you suggest please?

Re: [resolved] unable to count file row count

Anonymous — Mon, 09 Jan 2017 03:34:49 GMT

Hi
There is a global variable that counts the total number of files are found in the directory.
((Integer)globalMap.get("tFileList_1_NB_FILE"))
Regards
Shong

Re: [resolved] unable to count file row count

Anonymous — Mon, 09 Jan 2017 04:50:56 GMT

Hi
There is a global variable that counts the total number of files are found in the directory.
((Integer)globalMap.get("tFileList_1_NB_FILE"))
Regards
Shong

Dear Shong,
I used the same global variable , but its giving me sequence number for every row.
ex: if there are 40 files in the folder .
it will give me sequence
1
2
3
4.
.
.
.till 40
I want 40 to display to every where instead of sequence.

Re: [resolved] unable to count file row count

Anonymous — Tue, 10 Jan 2017 06:16:04 GMT

Where do you use this global variable? It should be used in next subjob if you want to get the total number of files, eg:
tfilelist--iterate...
|
onsubjobok
|
tjava
on tJava, print the value of this global variable.