topic Talend Big Data - How to access Map Reduce job logs from talend in Talend Studio

Talend Big Data - How to access Map Reduce job logs from talend

Anonymous — Sat, 16 Nov 2024 10:17:26 GMT

Hi,

I am new to talend. We are using talend for BigData. When we use hive component in the job it is submitted the map-reduce job and producing the result. But what we don’t know is how get the log information through talend job. Every time to see information we are going to Hadoop cluster, find the job and looking into logs. I would like to know is there any way we can get the full log or in worst case is there a way to get the Map Reduce Job ids/URLs?

Thanks for your help.

Re: Talend Big Data - How to access Map Reduce job logs from talend

Anonymous — Wed, 02 Nov 2016 07:19:35 GMT

Hi,

Could you please try to enable log4j level in advanced setting of "Run" tab to see if there is map-reduce job log printed on console?

Best regards

Sabrina

Re: Talend Big Data - How to access Map Reduce job logs from talend

Anonymous — Wed, 02 Nov 2016 16:51:05 GMT

I tried that but no luck. Actually there are multiple map reduce jobs that got triggers in "Main" flow of hive components. But I am not able to see what MapReduce jobs are getting created.

Re: Talend Big Data - How to access Map Reduce job logs from talend

Anonymous — Thu, 03 Nov 2016 07:04:27 GMT

Hi,
Could you please show us your " log4j level" setting screenshot?
Best regards
Sabrina