exec a sql statement in a tBigQuerySQLRow that load data from Parquet file to a BiqQuery table.
The table has some DATE data-type fields like these "yyyy-MM-dd" and if I check the schema of the parquet file, I see that the fields are of type Date (see screen attached). But when I check the parquet file the data type are transformed into timestamps and the bigquery table has timestamp fields too.
My question: is there a solution to write the field in Date rather than Timestamp?
Your database is probably holding the dates with their timestamps, but you are viewing them without their timestamps. People often confuse what their query analyser application shows with what the DB is holding. When you create your Parquet file, you will need to format the dates appropriately so that the timestamps are not included. Can you show the job that creates your Parquet file?
I have looked into this and I see the issue. The only type of "date" field that is used in Talend is the Java Date class. This always contains time. It holds everything. The formatting of Date columns simply deal with the human readable presentation of that data. The column itself does not lose any data. As such, in Parquet it will be creating a Timestamp field.
There is a feature request that I have found for this. But it is not ready for implementation yet.
it is really a pity that there is no solution. The only one I can think of is to copy the data in bigquery to a new table by making the appropriate casts.
