topic Re: Multiupload in AWS S3 in Talend Studio

Multiupload in AWS S3

Anonymous — Fri, 12 May 2017 14:05:58 GMT

Hi,

I am trying to load some data from MySQL to Redshift.

I am using tMySQLInput --> tRedshiftOutputBulkExec

Is there a way to enable the multiupload option of S3 in the component? The files that I am trying to load are bigger than 5GB and the jobs are failing.

Thanks!

Re: Multiupload in AWS S3

vapukov — Fri, 12 May 2017 14:49:14 GMT

are You already enable compression?

Re: Multiupload in AWS S3

Anonymous — Fri, 12 May 2017 14:51:14 GMT

No I hadn't used that. How does it work? It just compresses the files?

Re: Multiupload in AWS S3

vapukov — Fri, 12 May 2017 14:54:27 GMT

it compress files (on the fly), and make transfer faster

I have Jobs which transfer bigger tables and all ok, test it

Re: Multiupload in AWS S3

Anonymous — Fri, 12 May 2017 16:21:42 GMT

it works for some bigger files it's true But still it doesnt enable the multiupload. I think it had to be configured in the S3 Config.