Talend 8 bigdata Batch - ETL running error

bah602807 — Fri, 15 Nov 2024 23:07:25 GMT

I am getting the error like below whenever I run the big data batch job, like below

Starting job Copy_of_test_fin_app_std at 23:56 13/03/2022.

WARNING: An illegal reflective access operation has occurred

WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/C:/Users/Downloads/Talend-Studio-20211109_1610-V8.0.1/configuration/.m2/repository/org/apache/spark/spark-unsafe_2.12/3.0.1/spark-unsafe_2.12-3.0.1.jar) to constructor java.nio.DirectByteBuffer(long,int)

WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform

WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations

WARNING: All illegal access operations will be denied in a future release

org.talend.bigdata.launcher.utils.BigDataLauncherException: Error write block for file '/FileStore/FileStore/talend_jars/jcl-over-slf4j-1.7.29.jar'

at org.talend.bigdata.launcher.fs.DatabricksFileSystem.sendFileBlock(DatabricksFileSystem.java:449)

at org.talend.bigdata.launcher.fs.DatabricksFileSystem.sendFileBlock(DatabricksFileSystem.java:413)

at org.talend.bigdata.launcher.fs.DatabricksFileSystem.copyFromLocal(DatabricksFileSystem.java:545)

at org.talend.bigdata.launcher.databricks.DatabricksJob.uploadJars(DatabricksJob.java:168)

at org.talend.bigdata.launcher.databricks.DatabricksJob.executeJob(DatabricksJob.java:518)

at dw_fin.copy_of_test_fin_app_std_0_1.Copy_of_test_fin_app_std.runClientJob(Copy_of_test_fin_app_std.java:796)

at dw_fin.copy_of_test_fin_app_std_0_1.Copy_of_test_fin_app_std.runJobInTOS(Copy_of_test_fin_app_std.java:566)

at dw_fin.copy_of_test_fin_app_std_0_1.Copy_of_test_fin_app_std.main(Copy_of_test_fin_app_std.java:445)

Caused by: javax.net.ssl.SSLProtocolException: Connection reset

at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:126)

at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)

at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)

at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:259)

at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:137)

at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1155)

at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1125)

at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:823)

at org.talend.bigdata.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)

at org.talend.bigdata.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)

at org.talend.bigdata.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)

at org.talend.bigdata.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)

at org.talend.bigdata.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)

at org.talend.bigdata.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)

at org.talend.bigdata.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)

at org.talend.bigdata.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)

at org.talend.bigdata.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)

at org.talend.bigdata.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)

at org.talend.bigdata.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)

at org.talend.bigdata.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)

at org.talend.bigdata.http.impl.execchain.RetryExec.execute(RetryExec.java:89)

at org.talend.bigdata.http.impl.execchain.ServiceUnavailableRetryExec.execute(ServiceUnavailableRetryExec.java:85)

at org.talend.bigdata.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)

at org.talend.bigdata.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)

at org.talend.bigdata.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)

at org.talend.bigdata.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)

at org.talend.bigdata.http.client.fluent.Request.internalExecute(Request.java:173)

at org.talend.bigdata.http.client.fluent.Executor.execute(Executor.java:262)

at org.talend.bigdata.launcher.fs.DatabricksFileSystem.sendFileBlock(DatabricksFileSystem.java:440)

... 7 more

Caused by: java.net.SocketException: Connection reset

at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)

at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)

at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:448)

at java.base/sun.security.ssl.SSLSocketInputRecord.decodeInputRecord(SSLSocketInputRecord.java:237)

at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:190)

at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:108)

... 31 more

[ERROR]: dw_fin.copy_of_test_fin_app_std_0_1.Copy_of_test_fin_app_std - TalendJob: 'Copy_of_test_fin_app_std' - Failed with exit code: 1.

Job Copy_of_test_fin_app_std ended at 23:57 13/03/2022. [Exit code = 1]

Please help me what the root cause and resolution, if have.

Thanks.

Re: Talend 8 bigdata Batch - ETL running error

Anonymous — Mon, 14 Mar 2022 05:25:45 GMT

Hello,

Could you please confirm if you are facing this issue in a single job or with all the jobs?

Did you make a post-call to the API URL with some action of verifying the SSL connectivity in your job?

More information about your big data batch job design and work flow will be preferred.

Best regards

Sabrina

Re: Talend 8 bigdata Batch - ETL running error

bah602807 — Mon, 14 Mar 2022 15:42:36 GMT

I recently installed talend studio 8 and then created a couple of simple big data batch jobs. Ran those jobs using spark configuration. I got all the same errors above. The standard job, however, has no issue to run.

I am not sure what the root cause is making the error.

Re: Talend 8 bigdata Batch - ETL running error

Anonymous — Tue, 15 Mar 2022 03:23:46 GMT

Hello,

It seems there is something wrong with your setting and configuration as you are facing this issue with all the spark jobs.

With your subscription solution, Could you raise this with support please?

In this way, our colleagues from support team will give you a remote assistance Via zoom meeting through support cycle with priority.

Best regards

Sabrina

topic Talend 8 bigdata Batch - ETL running error in Talend Studio

Talend 8 bigdata Batch - ETL running error

Re: Talend 8 bigdata Batch - ETL running error

Re: Talend 8 bigdata Batch - ETL running error

Re: Talend 8 bigdata Batch - ETL running error