SSL Broken Pipe Error on tBigQuery Output

[Anonymous]

Hi all,

I have the following simple pipeline: to transfer 1000 rows of fake data from a row generator to a BigQuery Table. The flow works fine for 100 rows of data but when I change the rowgenerator to load 1000 rows into Big Query, I get a very confusing errors caused by:

a. org.jets3t.service.ServiceException: Request Error: Broken pipe (Write failed)

b. Caused by: javax.net.ssl.SSLException: Broken pipe (Write failed)

c. Caused by: java.net.SocketException: Broken pipe (Write failed)

Exception in component tBigQueryOutput_2_tBQBE (TestLoad)

org.jets3t.service.ServiceException: Request Error: Broken pipe (Write failed)

at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:563)

at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:264)

at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestPut(RestStorageService.java:1017)

at org.jets3t.service.impl.rest.httpclient.RestStorageService.createObjectImpl(RestStorageService.java:1828)

at org.jets3t.service.impl.rest.httpclient.RestStorageService.putObjectWithRequestEntityImpl(RestStorageService.java:1749)

at org.jets3t.service.impl.rest.httpclient.RestStorageService.putObjectImpl(RestStorageService.java:1741)

at org.jets3t.service.StorageService.putObject(StorageService.java:840)

at org.jets3t.service.impl.rest.httpclient.GoogleStorageService.putObject(GoogleStorageService.java:416)

at smartdatawarehouse.testload_0_1.TestLoad.tRowGenerator_1Process(TestLoad.java:873)

at smartdatawarehouse.testload_0_1.TestLoad.runJobInTOS(TestLoad.java:1294)

at smartdatawarehouse.testload_0_1.TestLoad.main(TestLoad.java:1143)

Caused by: javax.net.ssl.SSLException: Broken pipe (Write failed)

at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:127)

at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:320)

at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263)

at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:258)

at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1016)

at org.apache.http.impl.io.AbstractSessionOutputBuffer.write(AbstractSessionOutputBuffer.java:184)

at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113)

at org.jets3t.service.impl.rest.httpclient.RepeatableRequestEntity.writeTo(RepeatableRequestEntity.java:275)

at org.apache.http.entity.HttpEntityWrapper.writeTo(HttpEntityWrapper.java:94)

at org.apache.http.impl.client.EntityEnclosingRequestWrapper$EntityWrapper.writeTo(EntityEnclosingRequestWrapper.java:110)

at org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:118)

at org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:266)

at org.apache.http.impl.conn.AbstractClientConnAdapter.sendRequestEntity(AbstractClientConnAdapter.java:239)

at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)

at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)

at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:684)

at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)

at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)

at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:319)

... 10 more

Suppressed: java.net.SocketException: Broken pipe (Write failed)

at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)

at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)

at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)

at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81)

at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:351)

... 29 more

Caused by: java.net.SocketException: Broken pipe (Write failed)

at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)

at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)

at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)

at java.base/sun.security.ssl.SSLSocketOutputRecord.deliver(SSLSocketOutputRecord.java:320)

at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1011)

... 26 more

Does anyone have any insight into why this is occurring?

Thanks in advance!

[nfz11]

How often are you committing your connection?  Can you post a screenshot with the details of the tBigQueryOutput component?  Thanks.