Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi,
Please suggest which file type is faster while importing data to qlikview for example .txt, .xlsx, xml, HTML, .xls or any other.
I know QVD is faster then all of the above mentioned format, but which one is faster after QVD.
I cant say for sure but I would think .txt would be the fastest after QVD's. Less for the ETL engine to parse over.
Granted XLS are easier to maintain, but speed should go to .txt.
i think so, there is no straight answer for ur question.
may be xml is faster after qvd.
Are we splitting hairs here? You'll only see any difference in really large data sets and even then the saving would most likely be insignificant. You'd probably spend more time prepping the file than you'd gain in efficiency saving.
hey I am also talking about the large data set i.e. 14 Milions record which i forgot to mentioned above.
If the exact faster format is known then I will ask that format from client.
thanks
Priyank
14 million is nice chunk. Then format will impact speed to some degree.
I think what may matter most is what format is most reliable format they can deliver? If txt in csv form is a reliable form they can give you then it may be a good option. XML in my opinion takes more parsing albeit minimal, but also allows for more control.
It may be best to take it in the form that it originally comes in to ensure data integrity.
I agree - all file-formats (xml, html, xlsx) with hierarchically structurs will be require more efforts to parse them and will be (much) slower. Especially xlsx is very slow - often the loading-records within the load progress window jumps in hundred steps and by a txt-file in hundred-thousands.
- Marcus
14 million rows should rule out both XLS - which has a limit of 65535 rows and XLSX which is ~1 million rows.
HTML and XML has more overhead than regular CSV or fixed record text. I think fixed record text should be fastest. But they will most probably diverge by only a few percent is my guess.
With this number of records, it makes sense to keep the data in a database rather than a file.