how to write from one Hive table to another table with row counts?

Anonymous — Sat, 16 Nov 2024 05:03:22 GMT

I have a partitioned table in one database Share_In. This table is partitioned on date. There is one file which stores different dates and the data of these dates may or may not exist in the share_in database table. I want to copy the data as it is from this shared_in db to another table in shared_out db based on the dates in the file. I have used tHiveRow to write into shared_out db but i am facing 2 issues:

1. Even if the date mentioned in the file does not have corresponding partition in the Shared_In db, the partition gets created in Shared_out db.

2. How do i count the number of rows for each partition in tHiveRow component.

Below is the query i wrote in tHiveRow:

"insert into "+ context.HIVE_OUT_Database + ".xva_crva_data_out partition
(effectivedate = "+(Integer)globalMap.get("row1.rundate")+")
select legalentity, cif, cva, dva, fca, currencycode, effectivedate from xva_crva_data where effectivedate ="+ ((Integer)globalMap.get("row1.rundate"))

Re: how to write from one Hive table to another table with row counts?

Anonymous — Fri, 30 Aug 2019 11:03:06 GMT

Hello,

You can take a try to use use tHiveInput instead of tHiveRow component and pass "Select count(*) from"+context.TableName.

Best regards

Sabrina

topic how to write from one Hive table to another table with row counts? in Talend Studio

how to write from one Hive table to another table with row counts?

Re: how to write from one Hive table to another table with row counts?