What is the order on which tFTPFileList components decides to iterate the file from FTP server

[Sanjay5]

Hi,

I am looking to get some info on the order on which tFTPFileList component decides to iterate the files from FTP server.

I have 3 files in the server with different timing.

File1     :   Apr 17  11:20

File2     :   Apr 17  13:20

File3     :   Apr 17  14:20

Which file would be pulled first?

When i run my job, files are not getting downloaded based on timing. 

I expect File1 to be pulled first. (First come first serve logic) But that is not happening.

I see "Order By" option in tFileList component but that is not there in tFTPFileList.

Any thoughts?

[manodwhb]

@stamrake,in tFTPFilelist you do not have "order by"  Option,so first try to get the files in local directory and then list the files using tFileList,where you can specify the order by.

[Sanjay5]

@manodwhb

Thanks Manohar for the idea, 

It's a good workaround however still I am not sure how do i know that which file got pulled to local directory first.

If FILE3 gets pulled to local directory first and if I sort by time, it means I would be first processing FILE3 instead of FILE1.

Regards,

Sanjay T

[manodwhb]

@stamrake,can you sort by modified date on descending order then you may get File1.

[manodwhb]

@stamrake,still do you have an issue?

[Sanjay5]

@manodwhb

Hi, Thanks for the idea.

This trick could give me FILE1 to be processed at first, but there is no guarantee for that, as we do not have sorting option in tFTPFileList.

For my use case i need to be certain of which file is getting processed first.

Regards,

Sanjay

[manodwhb · Accepted solution]

@stamrake,in tFTPFile list we do not have option to sort, since you need to get the files in to local directory,from there you need to list using tFilelist. where in tFilelist you can specify the order by filename or file size or modified date or by default with acesnding or desending.

[Sanjay5]

@manodwhb

Hi,

I agree with your suggestion for the work around.

Guess what, I have requested Talend Enterprise team to have a feature request on the component tFTPFileList to have "Order By" Option.

Feature request detail: (TDI-40435)