Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Announcements
Join us in Toronto Sept 9th for Qlik's AI Reality Tour! Register Now
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable
‎2016-08-10 05:37 AM

[resolved] Talend Open Studio for Big Data - Spark components not showing up

Hi
  I downloaded Talend Open studio for Big Data Version 6.2.1 today and was looking to configure a Spark Job. I have a complete Hadoop environment on my laptop with 3 nodes in the Cluster. This includes a Spark Cluster as well. Presently i am writing scala code using Intellij and running my jobs. They are working quite well.
In this context, i would like to see if i can, instead of creating Scala code, use talend to work with my Spark Cluster. 
However after installing i don't know how to get the following
1. Do not see "Big Data Batch" under Job Designs
2. Do not see Spark as an option under Big Data in the Component palette 
Please let me know if there are some documentation to configure Spark with Talend
Regards
Balaji Krishnan
Labels (2)
Labels
439 Views
0 Likes
1 Solution

Accepted Solutions
Anonymous
Not applicable
‎2016-08-10 08:08 AM
Author

Thanks Sabrina.. Please close this post. I understand that the spark interface is more a paid option on top of the free community edition. Thank you

View solution in original post

439 Views
0 Likes
5 Replies
Anonymous
Not applicable
‎2016-08-10 06:10 AM
Author

Hi,
Here is bigdata product matrix page:https://www.talend.com/products/big-data. Could you please take a look at it to choose a Talend Big Data Integration solution with the feature set and licensing options to best fit your project and budget?
 Batch Processing (MapReduce, Spark), Native Hadoop Connectors and Real-Time Processing (Spark Streaming) are available in Talend subscription version not open source.
Best regards
Sabrina
439 Views
0 Likes
Anonymous
Not applicable
‎2016-08-10 08:08 AM
Author

Thanks Sabrina.. Please close this post. I understand that the spark interface is more a paid option on top of the free community edition. Thank you
440 Views
0 Likes
Anonymous
Not applicable
‎2016-08-11 06:39 AM
Author

Hi,
Feel free to let us know if there is any further help we can give.
Best regards
Sabrina
439 Views
0 Likes
Anonymous
Not applicable
‎2017-02-03 10:08 AM
Author

We are planning to buy Talend Bigdata licence. please help  !!
Do we have any component in Talend Bigdata SandBox to write custom spark code using Scala?  Or do I need to use any other Talend product to create BigData streaming and Batch Jobs in which I can write custom code also?? please reply thanks in advance.
it's urgent.
439 Views
0 Likes
Anonymous
Not applicable
‎2017-02-03 01:27 PM
Author

From what I've seen, custom spark code must be done via Java (java api to work with RDDs and pairRDDs and also dataframes api can be used).  When creating a spark job, the custom code can be put in tJava and tsqlrow.  Both of those components have comments for how to call the RDDs and DataFrames from your upstream inputs.
439 Views
0 Likes