We have a qlikview load process that is responsible for generating multiple qvd files based on a date. Each file containing only one days worth of data. The process will generate daily files from the last time it was run. What I have noticed is that when we are generating more than 1 file the size of the qvd file was growing even though the number of records was pretty static. File size started a 7k and would increase by about 3k per file.
What I found is that the QVD file is storing lineageinfo as part of the file. This lineageinfo in our case is the SQL statement used to produce the QVD. As the load progresses this lineageinfo grows as it adds new SQL statements.
We are running V11 SR13. Has anyone noticed this behavior and is there a way to turn this off.
Below is a sample of the QVD lineage info data.
First run.
- <Lineage>
- <LineageInfo>
<Discriminator>ODBC;DSN=AUTOQRY</Discriminator>
<Statement>DelayedBodies: SQL With DeptProdOFF (Fields) as (Select Fields FROM Tables WHERE DATE = '2004-04-02' GROUP BY Fields ) SELECT fields FROM Tablese GROUP BY Fields HAVING COUNT(DISTINCT field)>1</Statement>
</LineageInfo>
- <LineageInfo>
<Discriminator>ODBC;DSN=DALQRY1</Discriminator>
<Statement>Concatenate (DelayedBodies) SQL With DeptProdOFF (Fields) as (Select Fields FROM Tables WHERE DATE = '2004-04-02' GROUP BY Fields ) SELECT fields FROM Tablese GROUP BY Fields HAVING COUNT(DISTINCT field)>1</Statement>
</LineageInfo>
- <LineageInfo>
Next Run
<Lineage>
- <LineageInfo>
<Discriminator>ODBC;DSN=AUTOQRY</Discriminator>
<Statement>DelayedBodies: SQL With DeptProdOFF (Fields) as (Select Fields FROM Tables WHERE DATE = '2004-04-02' GROUP BY Fields ) SELECT fields FROM Tablese GROUP BY Fields HAVING COUNT(DISTINCT field)>1</Statement>
</LineageInfo>
- <LineageInfo>
<Discriminator>ODBC;DSN=AUTOQRY</Discriminator>
<Statement>DelayedBodies: SQL With DeptProdOFF (Fields) as (Select Fields FROM Tables WHERE DATE = '2004-04-03' GROUP BY Fields ) SELECT fields FROM Tablese GROUP BY Fields HAVING COUNT(DISTINCT field)>1</Statement>
</LineageInfo>
- <LineageInfo>
<Discriminator>ODBC;DSN=AUTOQRY</Discriminator>
<Statement>DelayedBodies: SQL With DeptProdOFF (Fields) as (Select Fields FROM Tables WHERE DATE = '2004-04-04' GROUP BY Fields ) SELECT fields FROM Tablese GROUP BY Fields HAVING COUNT(DISTINCT field)>1</Statement>
</LineageInfo>
- <LineageInfo>
<Discriminator>ODBC;DSN=DALQRY1</Discriminator>
<Statement>Concatenate (DelayedBodies) SQL With DeptProdOFF (Fields) as (Select Fields FROM Tables WHERE DATE = '2004-04-02' GROUP BY Fields ) SELECT fields FROM Tablese GROUP BY Fields HAVING COUNT(DISTINCT field)>1</Statement>
</LineageInfo>
- <LineageInfo>
<Discriminator>ODBC;DSN=DALQRY1</Discriminator>
<Statement>Concatenate (DelayedBodies) SQL With DeptProdOFF (Fields) as (Select Fields FROM Tables WHERE DATE = '2004-04-03' GROUP BY Fields ) SELECT fields FROM Tablese GROUP BY Fields HAVING COUNT(DISTINCT field)>1</Statement>
</LineageInfo>
<LineageInfo>
<Discriminator>ODBC;DSN=DALQRY1</Discriminator>
<Statement>Concatenate (DelayedBodies) SQL With DeptProdOFF (Fields) as (Select Fields FROM Tables WHERE DATE = '2004-04-04' GROUP BY Fields ) SELECT fields FROM Tablese GROUP BY Fields HAVING COUNT(DISTINCT field)>1</Statement>
</LineageInfo>
