Extract sample rows from delimited input file

[JZMatrix1]

I'm trying to accomplish something similar to what is in https://community.talend.com/s/feed/0D73p000004kMH3CAM, using TOS 7.3, and it seems as though the "Extract Line at Random" is no longer available for the tFileInputDelimited component. The closest option I've been able to find is the tReservoirSampling component, which isn't available in TOS. Does this functionality still exist within TOS 7.3?

** I've been away from using Talend for a number of years, so finding the difference in 7.x since my last 5.x/6.x days **

Thank you,

Jason

[gjeremy1617088143]

Hi, @Jason Ziemba​ 

I use a Talend data Services platform 7.3 and I can find the extract lines at random in the advanced settings of the tFileInputDelimited.

Send me love and kudos

[JZMatrix1]

My IDE is the Talend Open Studio for Data Integration (7.3.1.20200219_1130). Below is the screenshot of my tFileInputDelimited, advanced tab. Unfortunately, the random line option seems to be omitted from this platform in the latest version.

[gjeremy1617088143]

I think, you can do this for example :

read the file sent it into a hash

get the globalMap NB_LINE generated by the fileInput component :

create a globalMap of type int with value :Numeric.random(1,the globalMap NB_LINE generated by the fileInput component)

then read the hash --> use tflowtoiterate--> use a tfilterRow with this condition (CURRENT_ITERATION == (your globalMap))

[gjeremy1617088143]

you can also :

read the file, add a new field with a sequence , put into a hash read the NB_line

create a globalMap of type int with value :Numeric.random(1,the globalMap NB_LINE generated by the fileInput component)

read the hash

use a tfilterRow with this condition (field with sequence == (your globalMap))

[JZMatrix1]

Thank you for these suggestions. I did a different project that required a bit of similar logic and used a combination of global variables and a JavaRow component to pull in random numbers, determine if it was within the desired range, and then acted on it. I could do the same with this implementation, but got excited when I read about a checkbox that did all of that 'out of the box'.