Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi,
I`m building a Big Data Streaming job and I need to use the tSetKeystore in order to use it in the tKafkaInputAvro component.
I want to run this job locally but I get following error.
Please can you advise what configurations should be done in order to make it work.
Thank you!
Error:
[WARN ] 18:53:32 org.apache.hadoop.util.NativeCodeLoader- Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[WARN ] 18:53:32 org.apache.spark.SparkConf- In Spark 1.0 and later spark.local.dir will be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in mesos/standalone and LOCAL_DIRS in YARN).
java.io.IOException: No FileSystem for scheme: C
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.spark.util.Utils$.getHadoopFileSystem(Utils.scala:1726)
at org.apache.spark.util.Utils$.doFetchFile(Utils.scala:662)
at org.apache.spark.util.Utils$.fetchFile(Utils.scala:459)
at org.apache.spark.SparkContext.addFile(SparkContext.scala:1435)
at org.apache.spark.SparkContext.addFile(SparkContext.scala:1384)
at org.apache.spark.api.java.JavaSparkContext.addFile(JavaSparkContext.scala:669)
Regards
Hello,
tSetKeystore component will look for the .jks file in a spark cluster in a Big Data Streaming job.
For more information, please have a look at this online component reference: TalendHelpCenter: tsetkeystore-properties-for-apache-spark-streaming
Best regards
Sabrina