Skip to main content
Announcements
SYSTEM MAINTENANCE: Thurs., Sept. 19, 1 AM ET, Platform will be unavailable for approx. 60 minutes.
cancel
Showing results for 
Search instead for 
Did you mean: 
Haresh
Contributor
Contributor

how to speed up or use parallel execution using tftpget when downloading files from the ftp site?

I am observing that when use tftpconnection->tftpftilelist->tftpfileproperties-tbufferoutput/input->tmap->tftpget (sort the entire list of files in the remote ftp site lets say if it has 3000 odd files,it will sort and pull only the latest date (1 day date range)recent files) but since the files are around 100 MB each,i am observing it is taking forever to download 100 bytes per 5 second and takes quite a while to download to the local.

Is there any way to make the file download faster using tftpget or by enabling parallel execution,but i am noticing that the button is switched off or grayed out.

Please suggest?0693p00000AHDrxAAH.png

2 Replies
Anonymous
Not applicable

Hello , First, which release are you using ?

You can set up parallelization for your job by right clicking on first component and select "configure parallelization".

On the first job row, you can selelct , on parallelization tab, nuber of parallel thread you need

 

But it might not improve if you reached some limits, such as QoS or bandwidth. To be sure, you need check with a ftp client to check maximum rate you can reach

 

 

Haresh
Contributor
Contributor
Author

I am using the talend cloud 7.2 version and i dont see an option when i right click the first or any component to enable or configure parallelization.

 

Can you attach a screenshot of how it looks like,so i can try to see if there is anyway it would show up for me

 

I could only see under a particular component,upon selecting ->advanced settings ->enable parallel execution but that feature is grayed out for some reason?