topic Re: how to speed up or use parallel execution using tftpget when downloading files from the ftp site? in Talend Studio

how to speed up or use parallel execution using tftpget when downloading files from the ftp site?

Haresh — Sat, 16 Nov 2024 01:08:44 GMT

I am observing that when use tftpconnection->tftpftilelist->tftpfileproperties-tbufferoutput/input->tmap->tftpget (sort the entire list of files in the remote ftp site lets say if it has 3000 odd files,it will sort and pull only the latest date (1 day date range)recent files) but since the files are around 100 MB each,i am observing it is taking forever to download 100 bytes per 5 second and takes quite a while to download to the local.

Is there any way to make the file download faster using tftpget or by enabling parallel execution,but i am noticing that the button is switched off or grayed out.

Please suggest?

Re: how to speed up or use parallel execution using tftpget when downloading files from the ftp site?

Anonymous — Tue, 10 Nov 2020 09:25:25 GMT

Hello , First, which release are you using ?

You can set up parallelization for your job by right clicking on first component and select "configure parallelization".

On the first job row, you can selelct , on parallelization tab, nuber of parallel thread you need

But it might not improve if you reached some limits, such as QoS or bandwidth. To be sure, you need check with a ftp client to check maximum rate you can reach

Re: how to speed up or use parallel execution using tftpget when downloading files from the ftp site?

Haresh — Wed, 11 Nov 2020 22:31:02 GMT

I am using the talend cloud 7.2 version and i dont see an option when i right click the first or any component to enable or configure parallelization.

Can you attach a screenshot of how it looks like,so i can try to see if there is anyway it would show up for me

I could only see under a particular component,upon selecting ->advanced settings ->enable parallel execution but that feature is grayed out for some reason?