Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Governance Dashboard 2.01 size limitation?

Dear all,

I am trying to use the GD 2.01 to analyse the lineage of our models. The problem I have is that the Lineage-data misses some models which are present in our environment. If I narrow down the load to one specific folder I notice that all models loaded into the Lineage table are 6-30MB all models not loaded into the Lineage table are 130-850MB.

I notice the missing files are present in the Filelist.

Does any of you know if the GD limits the loading of lineage information to files smaller than a certain threshold? I tried to find it myself in the script, but I can't find any.

Kind regards,

Ricardo

1 Solution

Accepted Solutions
Tyler_Waterfall
Employee
Employee

I am not aware of any limitations to the file size of a QVW or QVD that can be "read" for lineage metadata.

Can you try reading the metadata of one of those larger QVWs manually?

To do this, create a new QV doc and in the load script click "Table File" to load. Then go to where the QVWs are and select "All Files" so you can see the QVWs. Select the QVW and in the load dialogue click on the "xml" radio button.

This way you are loading just the XML part of the QVW.

You should be able to save your app and reload it, seeing a bunch of metadata for that QVW.

View solution in original post

4 Replies
Tyler_Waterfall
Employee
Employee

I am not aware of any limitations to the file size of a QVW or QVD that can be "read" for lineage metadata.

Can you try reading the metadata of one of those larger QVWs manually?

To do this, create a new QV doc and in the load script click "Table File" to load. Then go to where the QVWs are and select "All Files" so you can see the QVWs. Select the QVW and in the load dialogue click on the "xml" radio button.

This way you are loading just the XML part of the QVW.

You should be able to save your app and reload it, seeing a bunch of metadata for that QVW.

Anonymous
Not applicable
Author

Hi Tyler,

it appears it is coincidence it are the biggest files in that single folder. As I analyse other files which are not read there appears to be no clear reason.

The thing I notice when loading the meta-data of the files is that they not contain any Lineage-info. I can't find a real common thing why no Lineage-info would be created. Some facts about the models:

- The models I am looking at are dashboards( and contain data, no dropped tables)

- The models have scheduled reload daily, when I reload them manually they produce Lineage-Info

- The models are of any sizes

- The models are made in different versions of Qlikview( from 8.x till most recent)

- The models are in different folders.

- I have admin rights in most of the models

One final thought, is it possible that after a Partial Reload, no Lineage Info is created?

Thanks for replying!

Tyler_Waterfall
Employee
Employee

Strange indeed.

Can you check a couple of the qvws that are not getting metadata read and see:

1- Are they Partial Reload?

2- Can you manually inspect the metadata in the qvw by opening the .qvw with a text editor?

Concerning the fact that the 'missing' metadata qvws showing up in the FileList - it is possible that the file ends up there because it is mentioned in reload log entry or a session log entry.

Anonymous
Not applicable
Author

Hi Tyler,

1. Yes, all of the files have a Daily Partial Reload and a weekly or monthly Full Reload( Partial reload is to update Section acces. Full Reload to update data)

2. Yes even without admin rights I can open all files with text editor or with Qlikview as xml.

I think it shows up in the file list as other meta-data is generated when doing a Partial Reload. I think only the Lineage Information is not generated.

In the meantime I made my own Lineage analyser which runs daily and keeps track of history. The hard thing is that we have 65+ models which all have a different schedule for full loads, so every day the situation is different. I tried to work around this by saving all Lineage in a History qvd

Every Day the following steps are executed:

1. Generate List with all files within the folders

2. use xml-load to recieve Lineage info for each file and Left-Join this on the file list ( now you have a list containing all files and where present Lineage Information)

3. Extract all File Names of which no Lineage info is found.

4. Retrieve Lineage information from Historical File for the list of step 3

5. Concatenate Lineage information from Historical with all Lineage info from step 2

6. Overwrite History File with table of step 5

By doing this daily, the historical file will build up the most recent, correct Lineage information.

Wrapping up: I think I managed to create my own Lineage Analyser. It is definitely not as elaborated as the Governance Dashboard or as the Lineage Dependency Analyser of Vasiliy Petrenko, but it does the job.

Thanks for the suggestions and help!