AWS S3 Connection behind proxy

[nchimakurty]

Issues: Not able to connect to AWS S3 bucket even bypassing proxy. Build job is 88 MB, I cannot attach in email, any other alternative to provide you job details.

I am able to connect to AWS S3 bucket successfully thru AWS CLI.

C:\Users\chimaku>set HTTP_PROXY=http://proxy.ops.tiaa-cref.org:8080

C:\Users\chimaku>set HTTPS_PROXY=http://proxy.ops.tiaa-cref.org:8080

C:\Users\chimaku>aws s3 ls tiaa-sds-dev-ue1-sdsdev-redshift-uds

                           PRE CLSS/

I am following below settings thru advanced settings also thru Talend preferences.

Thanks

Starting job SJ_S3_To_LocalHost at 17:03 17/03/2020.

[statistics] connecting to socket on port 3776

[statistics] connected

Unable to execute HTTP request: http://proxy.ops.tiaa-cref.org

[ERROR]: eds_poc.sj_s3_to_localhost_0_1.SJ_S3_To_LocalHost - tS3BucketExist_1 - Unable to execute HTTP request: http://proxy.ops.tiaa-cref.org

Exception in component tS3BucketList_1 (SJ_S3_To_LocalHost)

com.amazonaws.SdkClientException: Unable to execute HTTP request: http://proxy.ops.tiaa-cref.org

      at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1134)

      at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1080)

      at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:745)

      at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719)

      at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701)

      at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669)

      at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651)

      at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515)

      at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4443)

      at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4390)

      at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4384)

      at com.amazonaws.services.s3.AmazonS3Client.listBuckets(AmazonS3Client.java:942)

      at com.amazonaws.services.s3.AmazonS3Client.listBuckets(AmazonS3Client.java:948)

      at eds_poc.sj_s3_to_localhost_0_1.SJ_S3_To_LocalHost.tS3BucketList_1Process(SJ_S3_To_LocalHost.java:776)

      at eds_poc.sj_s3_to_localhost_0_1.SJ_S3_To_LocalHost.tS3BucketExist_1Process(SJ_S3_To_LocalHost.java:668)

      at eds_poc.sj_s3_to_localhost_0_1.SJ_S3_To_LocalHost.tS3Connection_1Process(SJ_S3_To_LocalHost.java:492)

      at eds_poc.sj_s3_to_localhost_0_1.SJ_S3_To_LocalHost.runJobInTOS(SJ_S3_To_LocalHost.java:1289)

      at eds_poc.sj_s3_to_localhost_0_1.SJ_S3_To_LocalHost.main(SJ_S3_To_LocalHost.java:1089)

Caused by: java.net.UnknownHostException: http://proxy.ops.tiaa-cref.org

      at java.net.InetAddress.getAllByName0(Unknown Source)

      at java.net.InetAddress.getAllByName(Unknown Source)

      at java.net.InetAddress.getAllByName(Unknown Source)

      at com.amazonaws.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:27)

      at com.amazonaws.http.DelegatingDnsResolver.resolve(DelegatingDnsResolver.java:38)

      at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)

      at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)

      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

      at java.lang.reflect.Method.invoke(Unknown Source)

      at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76)

      at com.amazonaws.http.conn.$Proxy8.connect(Unknown Source)

      at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:389)

      at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)

      at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)

      at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)

      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)

      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)

      at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)

      at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1256)

      at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1072)

      ... 16 more

[FATAL]: eds_poc.sj_s3_to_localhost_0_1.SJ_S3_To_LocalHost - tS3BucketList_1 Unable to execute HTTP request: http://proxy.ops.tiaa-cref.org

[Anonymous]

Have you tried using the tSetProxy component? It is described here: https://help.talend.com/reader/tnKcYfR9z1XfxISLACaZ~w/UkuAqfMTdgeFJEQmfWzfIQ

[pprevel]

Bonjour, 

j'ai le même problème, si vous avez une solution je suis preneur