Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik and ServiceNow Partner to Bring Trusted Enterprise Context into AI-Powered Workflows. Learn More!
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

TOS 6.4 with HDP 2.6 not finding libraries for some componentes (Pig, Sqoop)

Hi,

 

I'm using TOS Big Data 6.4. We've upgraded from HDP 2.5 to HDP 2.6 and many componentes that worked fine under HDP 2.5, are not working anymore. When you select the distribution HDP 2.6, it seems that Talend is not able to recognize which libraries to load, so it displays an error saying 'library_name cannot be resolved to a type'. However, if you select HDP 2.5, it uses the correct libraries (which are the same for both distributions) and it works properly.

So it seems like a bug in TOS 6.4 when you choose HDP 2.6

 

Any solution for this issue?

 

Thanks

Labels (2)
5 Replies
Anonymous
Not applicable
Author

Hello,

Talend studio doesn't prompt you to download the libraries needed? Can you see the required jar from module? Does this issue only repro on pig, sqoop components?

Best regards

Sabrina

Anonymous
Not applicable
Author

Hi Sabrina,

 

there are some componentes (hive) that recognize which libraries to use but this is not the case for sqoop and pig (these ones are the only ones which I found that doesn't work correctly until now). I can see the required jars from Module because, with Pig for example, it's the same version (0.16.0) for both distributions (HDP 2.5 and HDP 2.6), if I choose HDP 2.5 it works but if I choose HDP 2.6 it doesn't. There's not the usual prompt which tells you which one is the required jar. Instead of that, a wizard is displayed (I attach a screenshot), but it doesn't tell you which one specifically. If I load the jar with a tLibraryLoad, the previous jars are read but a new wizard is displayed saying more jars are missing and if I load them again with a new tLibraryLoad, new jars missing appear, it seems it never ends!

 

Regards

 

 


pig.png
Anonymous
Not applicable
Author

Hi,

Are you using talend open studio bigdata 6.4.0? We will make a testing on your issue to see if it repro on our side.

Thanks for your time.

Best regards

Sabrina

 

Anonymous
Not applicable
Author

Hi,

yes, that's the version we're using.

 

Thanks a lot.

 

Regards

Anonymous
Not applicable
Author

Hello,

Here is a related jira issue:https://jira.talendforge.org/browse/TBD-5215.

It seems this issue is fixed on V 6.4.1. Could you please take a look at it?

Best regards

Sabrina