How to use hive SQL context and spark sql context in tsqlrow of Talend big data job

Anonymous — Sat, 16 Nov 2024 07:58:11 GMT

Hi Experts,

I am new to Talend big data job (Spark). I am trying to read data from thiveinput and generating a file but facing below error. The same cluster connections are working fine in data integration job but not working in big data job. In addition, are there any documents available to use Spark components in Talend (like tSQLrow for hive and Spark context,thiveinput, etc)?

[WARN ]: org.apache.spark.util.Utils - Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
[ERROR]: org.apache.spark.SparkContext - Error initializing SparkContext

Thanks,

Punnam

Re: How to use hive SQL context and spark sql context in tsqlrow of Talend big data job

Anonymous — Fri, 13 Jul 2018 07:58:12 GMT

Hello,

We are supposing that you are aware that each component has pieces of information indicating the families they belong to and the conditions for their eventual availability.

The search on Talend Help Center can help on this. As you arer using Talend BigData solution, here are the different lists for Big data Jobs on different frameworks in your product

Spark Batch: https://help.talend.com/search/all?query=*+properties+in+Spark+Batch+Jobs&filters=EnrichVersion~%25226.3%2522

Spark Streaming: https://help.talend.com/search/all?query=*+properties+in+Spark+streaming+Jobs&filters=EnrichVersion~%25226.3%2522&content-lang=en-US

Let us know if it is OK with you.

Screenshots of your bigdata job setting screenshots will be preferred.

Best regards

Sabrina

topic Re: How to use hive SQL context and spark sql context in tsqlrow of Talend big data job in Talend Studio

How to use hive SQL context and spark sql context in tsqlrow of Talend big data job

Re: How to use hive SQL context and spark sql context in tsqlrow of Talend big data job