Qlik-Sense: QVD's and Inline Loads

[Anonymous]

Hi,

I am using Inline-Loads inside the data load editor to load and store data.

This looks like:

[$(aTable)]:

    LOAD

        *

    INLINE

        [ '$(header)'

    '$(value)'];

STORE

    $(aTable)

    INTO

[$(aPath).qvd] (qvd);

At the beginning of the data load editor script the code above works very well.

Unfortunatelly it somehow makes a difference how much code in the data load editor has been processed (why?).

The qvd saves for some reason the following data:

<Lineage>

     <LineageInfo>

          ...

     </LineageInfo>

<LineageInfo>

This Lineage Info is getting more and more as the data load editor script is processed.

This does for once store data that should not be stored (making a 5KB file into a 100KB+ file),

it also reveals code (For example SQL scripts) from the loading script for no reason I can imagine .

My question therefor is why LineageInfo is stored into qvds and how to negate this.

Is this caused by the script above or by some qlik-sense setting?

[Anonymous]

When I paste the qvd into word, word tells me the it has over 3000 lines.
Quite a lot for storing a single value.

[marcus_sommer · Accepted solution]

To have the lineage-data is often quite useful to track from where the data are coming. By loading data through large loops it could get quite heavily and especially if the loaded data are relative small the lineage-data could becomes much bigger then the real data.

If there is really no other practically way to load and store the data you could disable this feature. Unfortunately not on document-level else global. A howto is described here:

Re: Expansion of QVD file size during iterated process

Edit: I just see that you are talking about Sense. I think it could be there disabled too, but don't know how.

- Marcus

[Anonymous]

Thank you for your reply Marcus,

so this would need to be set globally rather than for a single app or even a single qvd-storage.

I see that lineage data can be useful especially when sqls are executed, in order to see how the data was created and manipulated.

If Tables are loaded inline with no reference to other existing data it is, in my oppinion, quite useless to save everything that happened to this point though. It would be convenient to have a keyword that could disable (or enable?) storing lineage data, I only realized that it exists when I noticed how it slowed my app down. What I can read in this data is also not very helpful, but I haven't looked into it too much yet.

[marcus_sommer]

What is the reason to store a bunch of single values each within an own qvd? Maybe there are better ways to reach the aim.

- Marcus

[Anonymous]

I am storing variables that I am loading from other apps.

The inline load is the only workaround I know of at the moment.

[marcus_sommer]

Ok. - variables but why in each in an own qvd? Why not all into a single file maybe with some additionally informations like comments and where and to what to use?

- Marcus

[Anonymous]

First off - I think it is a good idea to also include a comment.

Second off - I like to have single files for single variables for those reasons:

1. The variables are not linked to each other and can / do exist on their own in their own usecase. A different app may only need one and not all of the variables.

2. The variables may update with different frequencies. I store a special variable once it changes its state, so I can not include other variables that may not even exist at that point.

3. It is an artistic decision how to structure data storage. I like the idea to have a folder with seperate files for each variable. Every file has exactly one purpose.

Unfortunatelly this is derailing a bit from how to solve it to why to do it in the first place.

I guess I will have to live with the extra data being stored, if not feel free to tell me.

[marcus_sommer]

If I would planning to develop and manage variabls externally I would tend to keep them within a centralized repository. Maybe you get here further ideas what might be the most suitable approach in your case:

Central KPI Formula Repository (solution)

Storing variables outside of QlikView

Loading Variables Via Loadscript

Managing Variables

Re: Setting a variable in script...

How to store and recover variables

Create Colour Variables from an Excel Workbook

- Marcus