extract data from oracle using spark job

Anonymous — Sat, 16 Nov 2024 05:09:14 GMT

hi,

i m trying to read the data from oracle table using big data spark job. I am using 6.2. I have standard job which is running fine and planning migrating to big data job.

I am using YARN client for the Spark Mode. When I am using only files it is working fine,but if I include oracle component, it is throwing below error.

For a test, i created a sample job as attached

Query I am using is select (*) cnt from schema.table where ABC=111;

In "Advance Spark and JDBC parameter" setting, added below

"dbtable" --> schema.table

Please guide me if I am doing something wrong or how to solve this?

org.apache.spark.sql.catalyst.parser.ParseException:
no viable alternative at input '<EOF>'(line 1, pos 0)

== SQL ==

^^^

at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:197)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:99)
at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:45)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parseTableIdentifier(ParseDriver.scala:48)

Re: extract data from oracle using spark job

Anonymous — Wed, 07 Aug 2019 08:41:45 GMT

Hello,

This issue seem to be the Spark Configuration, Use local mode property settings. Could you please try to select YARN client for the Spark Mode in the Spark Configuration of the Job? Let us know if it helps.

Best regards

Sabrina

Re: extract data from oracle using spark job

Anonymous — Fri, 09 Aug 2019 22:22:31 GMT

Hi @xdshi,
I am already using YARN Client. I request you please go thru my query again.
Even after using YARN client, I am facing this issue.

topic Re: extract data from oracle using spark job in Talend Studio

extract data from oracle using spark job

Re: extract data from oracle using spark job

Re: extract data from oracle using spark job