Load a large file using tloop

[AJafri]

I have file with million records need to load it in multiple iteration using tLoop. File have column with sequential value in it. We can load 10000 records in one iteration. I am stuck in the logic that has to be used in tLoop

[konvertilo]

Hello

You need to give more context. Why do you want to use tloop ? there is a tFileInputDelimited that can read huge file, you don't have to build chunk of data.

What is the usage of the sequential value in this file ?

[Anonymous]

Hello,

Could you please elaborate your case with an example with input and expected output values? Which will be helpful for us to understand your situation.

Best regards

Sabrina

[jeoste]

Hello,

What you can do is from your tFileInput split it into X different files (of 10 000 lines as mentioned).

Then with tFileList, you can read each file and insert their content int DB. In this case no need to use tLoop, tFileList is replacing this behavior

More info here : https://stackoverflow.com/questions/10348570/split-file-to-more-files-in-talend