tFTPGet - Super Slow

[talendtester]

1. Is there a component faster than tFTPGet?

2. Is there some setting which can make the tFTPGet transfer speed faster?

3. Is there some config setting which will improve the transfer speed?

I am using TOS Version: 7.1.1

We have daily files which are 34GB compressed to FTP from an AWS external server which I am told is not limited.

Using the tFTPGet component I can get the file, the issue is the transfer speed is less than 1MB per second.

When I transfer the files via CyberDuck I am seeing speeds of 7.8MB per second, so it takes around an hour per file.

[vapukov]

hi,

there are no special settings for this component 

you could test with command line scp (winscp in case of windows), if it will be also faster - then use tSystem for call command line utility

[talendtester]

With winSCP I am getting between 1.5MB and 2MB per second.

[vapukov]

as I understand CyberDuck support multithread for downloading (multiple-connections-for-file-transfers )

winscp - not

so you can check by searching command line clients like - https://aria2.github.io/

or test command line version of CyberDuck - https://duck.sh/

[talendtester]

I used tLibraryLoad to load edtftpj-2.0.4.jar then in tJava:

((com.enterprisedt.net.ftp.FTPClient)globalMap.get("conn_tFTPConnection_1")).setNetworkBufferSize(65536);
((com.enterprisedt.net.ftp.FTPClient)globalMap.get("conn_tFTPConnection_1")).setTransferBufferSize(65536);

Error:

Detail Message: The method setNetworkBufferSize(int) is undefined for the type FTPClient

[vapukov]

why you manually load library, if it already used by component?

and why you are use libray from 2009?!!

Version 2.0.4
(13 February 2009)

[talendtester]

If I don't I am getting this error:

Detail Message: com.enterprisedt cannot be resolved to a type

I downloaded edtftpj-2.5.0 to use with tLLibraryLoad, now I get:

java.lang.ClassCastException: com.jcraft.jsch.ChannelSftp cannot be cast to com.enterprisedt.net.ftp.FTPClient

[vapukov]

my question was - why you are use outdated library?

in any case starting from begin of the thread, I not sure increasing only this parameter resolve your problem

Duck work fast because run several threads in parallel, in description of library I not found nothing about this 

[talendtester]

The duck.sh website is blocked by our firewall.

Is this what you are suggesting?
https://blog.cyberduck.io/2015/06/24/scripting-cloud-storage-copy-using-the-command-line-interface-c...

[vapukov]

yes, but just as variant

search google - multithread FTP command line client and test what better for you