QVW LineageInfo missing

[blaise]

Hello fellow Qv'ers

Im trying to retrieve the qvw data lineage info from all my QVW but for some reason this data isn't present in the QVW when loading into QV as XML.

The lineage info resides in (XmlSimple, Table is [DocumentSummary/LineageInfo]).

Does anyone know why some qvw's doesn't have this info? I also have a lot of QVW's that also miss the DocumentSummary/FieldDescription and DocumentSummary/TableDescription.

Br Johan

[Tyler_Waterfall]

I think this might be due to the reload logic of the GovDB which it doesn't appear accounts for the Discriminator element in the QVD lineage metadata. I'll need to take a look closer.

[Tyler_Waterfall]

Okay, so it does appear that the Governance Dashboard does not check for the <Lineage> element in the QVD metadata xml, even though when AllowDataLineage = 1 is configured, you do get database connection and statements in the QVD xml metadata, like:

     <LineageInfo>

       <Discriminator>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\Data\ABC_1.0.mdb;Mode=Share Deny None;Extended Properties=&quot;&quot;;Jet OLEDB:System database=&quot;&quot;;Jet OLEDB:Registry Path=&quot;&quot;;Jet OLEDB:Database Jet OLEDB:Engine Type=5;Jet OLEDB:Database Locking Mode=1;Jet OLEDB:Global Partial Bulk Ops=2;Jet OLEDB:Global Bulk Transactions=1;Jet OLEDB:New Database Jet OLEDB:Create System Database=False;Jet OLEDB:Encrypt Database=False;Jet OLEDB:Don't Copy Locale on Compact=False;Jet OLEDB:Compact Without Replica Repair=False;Jet OLEDB:SFP=False</Discriminator>

       <Statement>LOAD CategoryID,

    ItemCost,

    ItemPrice,

    ProductID,

    ProductName,

    QuantityPerUnit,

    SupplierID,

    UnitCost,

    UnitPrice,

    UnitsInStock,

    UnitsOnOrder;SQL SELECT *

FROM products</Statement>

     </LineageInfo>

However, there is a limitation here in that the QVD will list all load/select statements and store ... into statements from the QVW that created it - not just the ones specifically related to it.  In other words, if you have a QVW which loads from four different database tables -- resulting in four distinct data tables in the QVW - and each of these is stored to QVD, all four QVDs will list all four data connections and load statements. So there is no programatic way that I know of to correctly specify which Load connection(s)/statement(s) apply to each QVD that is generated.

In summary - there are two limitations in play:

1- QVD metadata (xml) includes all lineage metadata from its CreatorDoc QVW, not just ones related to the table stored to QVD.

2- Governance Dashboard does not capture the Discriminator and Statement lineage data from QVDs (only the CreatorDoc).

[Anonymous]

Hi Tyler (twa),

Thank you for your reply.

The thing that QVD includes all lineage metadata from its CreatorDoc QVW is most likely a bug and I think you should raise it.

It's very strange that QVW includes lineage information per data model table and the same hasn't been applied to QVD.

Are you going to discuss that with R&D team?

Best regards,

Janusz

[Anonymous]

Hi All,

I've just noticed a release of the new QGD version 2.0.2.

QlikView Governance Dashboard 2.0.2 is now available

Will check if it fixes the lineage issue.

Thanks,

Janusz

[Anonymous]

Hi All,

Nope, the same wrong behaviour happens in QGD 2.0.2.

Thanks,

Janusz

[Tyler_Waterfall]

No changes to the GovDB regarding lineage handling were made in 2.0.2. Don't forget to check the release notes here:

[rwunderlich]

Tyler,

The same limitation re the coarseness of the <LineageInfo> applies to QVWs as well. This is just the way the product works. That is, all sources up to that point up in the script will be reported as sources. I would love to see true field lineage in the product. But the current state is that both QVW and QVD lineage include all data sources loaded up to that point in the script, without discrimination as to if the source actual contributed to the specific output.

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

[mla3eruva]

I've found where to turn this on for the desktop and when I manually reload the QVDs I can now see the lineage in the QVD header.   We recently upgraded our server environment to 12.1 and now I'm not seeing the header info for all the QVDs that are generated nightly.   Is there a setting we need to make on the server and if so, where do I find that?

Thanks in advance.

[rwunderlich]

The location for the settings.ini file for server reloads is:

C:\Windows\system32\config\systemprofile\AppData\Roaming\QlikTech\QlikViewBatch

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

[mla3eruva]

Thanks.  I found another thread that said setting should be -1, but the setting for desktop is 1.   Which is correct?