Solved: Question about QVD related best practices - Qlik Community

Report Inappropriate Content · ‎2016-02-24

Hi Qlik-community,

A new QlikView customer asked me the following question. What practice do you follow?

Is it good practice to load all records from tables into qvd files for future development? Or it is better just to keep in qvd subset of data that required for current development.

For example, we have some tables with 142 mln and 50 mln records and we need to refresh them every day.

Thanks,

Ventsi

johnw · ‎2016-02-24

I'm of mixed opinion on this one due to the size of the tables. I think I've only ever loaded data I actually need, except occasionally from very small tables. And for very big tables, I certainly use incremental loads, and recommend doing so.

But that creates an annoyance if you need to add a field. Generally, you're going to want that field applied to all the old data as well, not just added on new rows. So you'll need to do a full reload every time you add a field. If you only add fields now and then, it may not be a big deal. But if you're just starting out, and you can spot 20 fields on the table that you think you're very likely to need at some point, maybe go ahead and put them there. Clean up in a year if you never use them, perhaps.

Or just do what I've done and others recommend, and only add fields as you need them.

View solution in original post

Gysbert_Wassenaar · ‎2016-02-24

I wouldn't load data that I might never use.

talk is cheap, supply exceeds demand

Anonymous · ‎2016-02-24

I am just going through a tranche of qvd generators and getting rid of fields that are not needed.

I wish these unwanted fields had never put in there "just in case".

maxgro · ‎2016-02-24

Is it good practice to load all records from tables into qvd files for future development? Or it is better just to keep in qvd subset of data that required for current development.

I usually load the records and fields I use, I can add records and datain future, if needed.

Sometimes, for very small tables, I load all.

For example, we have some tables with 142 mln and 50 mln records and we need to refresh them every day.

For this, only the data you need. And try with an incremental laod.

johnw · ‎2016-02-24

I'm of mixed opinion on this one due to the size of the tables. I think I've only ever loaded data I actually need, except occasionally from very small tables. And for very big tables, I certainly use incremental loads, and recommend doing so.

But that creates an annoyance if you need to add a field. Generally, you're going to want that field applied to all the old data as well, not just added on new rows. So you'll need to do a full reload every time you add a field. If you only add fields now and then, it may not be a big deal. But if you're just starting out, and you can spot 20 fields on the table that you think you're very likely to need at some point, maybe go ahead and put them there. Clean up in a year if you never use them, perhaps.

Or just do what I've done and others recommend, and only add fields as you need them.