Spark Streaming to hive

[Anonymous]

Hi All,

I am new to talend. I am trying to build a job which will connect to mapr stream , consume the data and write the data to a hive table. 

I am using maprstream input to consume the data which is consuming the data correctly. but when inserting the data into hive using thiveoutput component its not working. I am even not getting any error. Can anybody help here.

Thanks,

Ranjit.

[Anonymous]

Are you Posting to partitioned Hive table (in Append Mode) ? can you please attach the screen shots of the Job and the configurations of the tHive output ?

[Anonymous]

Hi,

No I have not selected partitioned option. And save mode is Append. I am using 6.4 Real time big data platform.

The job is a big data streaming job, with one component tMapRStreamInput Input and the other one is tHiveOutput.

I can see the messages are getting consumed using the tMapRStreamInput, but I can't insert them to hive tables.

Note: Cant attach the screenshot because of organisation security policy.

Thanks,

Ranjit.

[Anonymous]

were the Hive tables created upfront before appending the data ? I have this issue when the Hive tables weren't created upfront and tried to append the data to it. 

[Anonymous]

Yes, I have created the hive tables before.The problem is it doesn't give any error as well.

[Anonymous]

can you able to print the data to the console ? if so, i don't see the reason why it won't get inserted into a Hive table.

By the explanation the only thing that can go wrong is the Hive table schema created upfront should match the parsing schema.

[Anonymous]

Yes, I am able to print the data in console. The only thing I am wondering i there are any schema mismatch error, then it should show in the console.

[Anonymous]

schema mismatch will print the data to the console and it doesn't post the data to the hive table, make sure the schema's are defined and i would prefer in using lower case letter's. Please post if that solved the issue after trying that

[Anonymous]

do you mean the tHiveOutput component will itself print the data in case its not able to insert the data. Then its not printing form me as well. I was different component to print the data.

[Anonymous]

Printing the data to the console is through tlog row. sorry for not being clear enough