Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Dear everybody,
I'm trying to achieve a mass conversion of QVD file to parquet file for a demo server.
Many of my QVD contain "mixed" data - ie numeric and text data in the same field which leads to a loss of data with parquet file
Storing fields with mixed data types into parquet may result in loss of data
[657748] values dropped in [MY_FIELD]
I can deal with mixed field one by one using the text() function but my purpose is to do a general load & Store with LOAD * from myqvdfile then store * to myparquetfile;
Does anybody have a solution ?
You may read the XML-Header of the QVD's which contain a lot of information about the field-data - at least numeric fields and string-fields are uniquely to identify in <NumberFormat> and <Tags>. By mixed fields I'm not sure but I think the reverse approach of not being a number or text should identify the mixed ones.
A bit simpler may be to load all fields as strings - maybe by using qvdnooffields() and qvdfieldnames() in a loop and creating an appropriate load-statement within a variable which then used in the final table-load:
Hello, any updates regarding this, we have the same issue here.