Name an output file with the number of lines of a tDBInput resultset

[Anonymous]

Hi all,

I've been using TOS for a couple of weeks, now comes the time to ask my first question to the community....

In my job I'm running a query on a MySQL database in order to extract a table content which is written into a csv file, via a tMap. Nothing uncommon, that's the easy part

Now I would like the target file to be named according to the number of lines of the resultset.

Like "blablabla_currentdate_lineCount.csv"

I know about the NB_LINE global variable BUT it can't be used for this purpose, because as I understood it, global variables are only available once the job is finished, and the output file is created during  the job. So my file ends up called "blablabla_null" ... 

Same problem if I involve the globalMap object to try and increment an integer on each row, it takes the first value (0, assigned in the prejob) and not the final one. (moreover in this last scenario, the resulting file is empty... But that's probably another issue)

Thank's a lot for your help, any suggestion would be greatly appreciated.

[Anonymous · Accepted solution]

Depending on the size of your result set, you can use the buffer components for this. You also can use the NB_LINE global variable instead of incrementing the count yourself.  The only restriction to using that is that it has to be after that entire subjob is complete to get an accurate count.

  So with this strategy you are effectively caching the results of the query in the buffer, that allows the subjob to finish, then in the next subjob, retrieving the results of the buffer, and the completed query result set count.

Edit: or you can just rename the file when the first subjob is done ¯\_(ツ)_/¯

[Anonymous]

Brilliant, exactly what was needed !
The fact is that you never really know what you can expect from a component before actually using it 😉
Thank you for sharing.