Data Studio: tBigQueryBulkExec fails to upload file to Google Storage

jamesasu — Sat, 16 Nov 2024 04:08:17 GMT

Hi,

I'm having issues uploading a JSON file to Google Storage (and ingesting to BigQuery). The error I'm getting is as follows:

Exception in component tBigQueryBulkExec_1 (Opps_to_BigQuery)
org.jets3t.service.ServiceException: Request Error: Broken pipe (Write failed)
	at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:563)
	at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:264)
	at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestPut(RestStorageService.java:1017)
	at org.jets3t.service.impl.rest.httpclient.RestStorageService.createObjectImpl(RestStorageService.java:1828)
	at org.jets3t.service.impl.rest.httpclient.RestStorageService.putObjectWithRequestEntityImpl(RestStorageService.java:1749)
	at org.jets3t.service.impl.rest.httpclient.RestStorageService.putObjectImpl(RestStorageService.java:1741)
	at org.jets3t.service.StorageService.putObject(StorageService.java:840)
	at org.jets3t.service.impl.rest.httpclient.GoogleStorageService.putObject(GoogleStorageService.java:416)
	at sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery.tBigQueryBulkExec_1Process(Opps_to_BigQuery.java:1820)
	at sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery.tSalesforceInput_1Process(Opps_to_BigQuery.java:1606)
	at sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery.runJobInTOS(Opps_to_BigQuery.java:2400)
	at sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery.main(Opps_to_BigQuery.java:2200)
Caused by: javax.net.ssl.SSLException: Broken pipe (Write failed)
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:127)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:320)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:258)
	at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1016)
	at org.apache.http.impl.io.AbstractSessionOutputBuffer.write(AbstractSessionOutputBuffer.java:184)
	at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113)
	at org.jets3t.service.impl.rest.httpclient.RepeatableRequestEntity.writeTo(RepeatableRequestEntity.java:275)
	at org.apache.http.entity.HttpEntityWrapper.writeTo(HttpEntityWrapper.java:94)
	at org.apache.http.impl.client.EntityEnclosingRequestWrapper$EntityWrapper.writeTo(EntityEnclosingRequestWrapper.java:110)
	at org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:118)
	at org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:266)
	at org.apache.http.impl.conn.AbstractClientConnAdapter.sendRequestEntity(AbstractClientConnAdapter.java:239)
	at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:684)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:319)
	... 11 more
	Suppressed: java.net.SocketException: Broken pipe (Write failed)
		at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
		at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
		at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
		at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81)
		at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:351)
		... 30 more
Caused by: java.net.SocketException: Broken pipe (Write failed)
	at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
	at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
	at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
	at java.base/sun.security.ssl.SSLSocketOutputRecord.deliver(SSLSocketOutputRecord.java:320)
	at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1011)
	... 27 more
[FATAL]: sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery - tBigQueryBulkExec_1 Request Error: Broken pipe (Write failed)

I'm testing this locally using Talend Studio 7.2.1.

The JRE that Studio is using is Java SE 8 [1.8.0_131].

I'm using interoperability access key and secret key associated with the main service account on this project and I have allocated "Storage Object Owner" role to the service account.

Any ideas on how I can get this to work?

Re: Data Studio: tBigQueryBulkExec fails to upload file to Google Storage

jamesasu — Fri, 22 Nov 2019 03:40:08 GMT

I ended up solving this error by using Java 11 instead of Java 8.

topic Re: Data Studio: tBigQueryBulkExec fails to upload file to Google Storage in Talend Studio

Data Studio: tBigQueryBulkExec fails to upload file to Google Storage

Re: Data Studio: tBigQueryBulkExec fails to upload file to Google Storage