Unexpected null schema column value returned by tHiveInput query in a Spark Job

[TalendSolutionExpert]

Talend Version	6.3.1
Summary
Additional Versions
Product	Big Data
Component
Problem Description	When querying a Hive table with a tHiveInput component in a Big Data batch Spark Job, an unexpected null column value is returned. To illustrate this issue, consider a Hive table named hivetable that has two columns, id and name, and one row: create table hivetable (id int, name string); insert into hivetable values (1,"one") A Spark Job is composed of the following tHiveInput and tLogRow components, as shown: When executing the Spark Job to query the hivetable table, the Name column has the value null, as shown by the tLogRow output: .--+----. |tLogRow_1| |=-+---=| |id|Name| |=-+---=| |1 |null| '--+----' You would expect the Name column value to be one instead of null.
Problem root cause	In the creation of the Avro payload related to the tHiveInput output, the column names are always lowercase. In this case, the issue is caused by the capitalized column name (Name) in the tHiveInput schema, which you can see in the screenshot. Note: You may have a similar issue when using components other than tLogRow, such as tFileOutputDelimited.
Solution or Workaround	Ensure that all schema column names in the tHiveInput component are lowercase.   In this example, when the tHiveInput schema column names id and name are lowercase, tHiveInput returns the expected row values: .--+----. |tLogRow_1| |=-+---=| |id|name| |=-+---=| |1 |one | '--+----' The name column value is now one instead of null.
JIRA ticket number