[resolved] tStatCatcher/tFlowMeter for file iterations
Hi,
If I use the stat catcher and/or flowmeter, and I am iterating through a set if input files, will the counts be available for each file or will the count be for the combined total for all files that are looped through?
Thanks in advance,
1) Delete the 'oncomponentok' link, see picture 1.
That's the way I did it at first, but then the RowMeterCatcher fires only after the subjob "Browse Files" finishes; as a consequence, the "Get Filename" mapper reads the file name as it is after subjob termination, i.e., the last file that was processed, and outputs this single filename for each line of statistics. So for instance instead of getting
file1 100 lines
file2 200 lines
file3 300 lines
I get
file3 100 lines
file3 200 lines
file3 300 lines
2) The tOracleOutputBulk and tOracleBulkExec components must be used together. see picture 2.
I don't understand this one too well. Wouldn't adding another OracleOutputBulk step introduce an additional, redundant, intermediate csv file? Currently the job fills up a csv file with the ouput from several input binary files then load the csv file in one go using sqlldr. This seems to work quite well as long as I don't introduce the statistics steps. Is there a better way to do that under Talend? Are the multithreading issues related to this architecture?
Thanks a lot for your help,
Chris
Hi there again,
sorry to insist but does anyone has any insight on the issue at hand? Is there a flaw in TOS multithreading support or should the job be designed differently?
Thanks to all for your help,
Chris
