tFileInputDelimited missing rows?

[Anonymous]

Hi
I have a subjob that processes some data and outputs it to a CSV using tOutputFileDelimited (1167 rows). I need to pull this into a tFileInputDelimited further down the flow, but every time it imported 0 rows.
So I split the subjob so that the tFileInputDelimited was the first component in a second subjob that was triggered on the first job completing, however now it's only pulling in 57 rows!
If I run the second subjob independently, it pulls in every row from the file, is there any reason it won't do this from onSubjobOk?
The job is essentialy like this:

tMap --> tFileOutputDelimited
        ¦
        v
onSubjobOk
        ¦
        v
tFileInputDelimited --> tMap

Any help would be appreciated as the output file is working fine and I can't under stand why the input won't read all the data rows unless the subjob is ran independently! I have attached images of the input and output settings...

[Anonymous]

You may be running into a timing issue where the output file has not yet been flushed to the drive and closed. If you want to keep the current design, put in something like a 30 second tSleep to give the data time enough to flush to disk.
However, a better solution is to not use an external file at all. For this very small number of rows, take a look at tHashOutput and tHashInput. This is an in-memory hash table that will give better performance, operates in real-time, and does not rely on an external drive system to function.
Be aware of things like memory consumption when using the tHash components. I have jobs that cache several hundred thousand rows of data and they are just fine. Your mileage may vary, depending on your environment.

[Anonymous]

hi all,
can we have a screenshoot of your job ?
regards
laurent

[Anonymous]

Hi wrlawton, I tried this with the tsleep component but only got 2 extra rows of data. I don't seem to be able to find thashinput or output?

[Anonymous]

I don't think is is a timing or flushing issue.
As kzone says, post a screenshot of your job