Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

LOAD from QVD / Duplicate rows in QVD vs. SQL

I have a much large QVW with multiple QVD references and I recently noticed that all the fact tables have duplicate rows when I load data from the QVD.

I created this quick script to simulate a typical QVD Store and load process. The SQL statement fetches one record from a SQL server table but as soon as I use the LOAD * statement, QV says there are two rows in the table.

If I add the DISTINCT clause to make it LOAD DISTINCT * there is only one row BUT why is this necessary. Do you always need to put DISTINCT in the LOAD statement from a QVD?

TBLRECORDS:





SQL SELECT * from SERecords where ID = 3165409;

STORE

TBLRECORDS INTO TBLRECORDS.QVD (qvd);

LOAD * FROM TBLRECORDS.QVD (qvd);

Thanks, Pete

Thanks, Pete

Thanks, Pete



1 Solution

Accepted Solutions
rwunderlich
Partner Ambassador/MVP
Partner Ambassador/MVP

Your SQL SELECT loads one row into the table. You store to a QVD. Then you read the QVD which adds another (same) row to the table.

View solution in original post

5 Replies
rwunderlich
Partner Ambassador/MVP
Partner Ambassador/MVP

Your SQL SELECT loads one row into the table. You store to a QVD. Then you read the QVD which adds another (same) row to the table.

Not applicable
Author

Thanks Rob. I guess I misunderstood the proper usage of LOAD and thought it was just to fetch records from the QVD into memory for use within the current QVW session.

I am working towards staging records in a separate loading QVW from SQL into the QVD files and then having a more heavily utilized QVW that doesn't fetch from SQL but only from those staged QVD files. I guess by having both the SQL with STORE and the LOAD I was doubling up records

thanks,

Pete

rwunderlich
Partner Ambassador/MVP
Partner Ambassador/MVP

Pete,

You understand correctly. The LOAD creates the table in the Loader QVW. The typical usage is to DROP the table after STORE so it doesn't occupy space in the Loader qvw disk file.

Anonymous
Not applicable
Author

Hi Rob,

Could you help me with loading?

I load data from SQL ( last month)

LOAD *;

SELECT * FROM dbo_***;

then I add historical data which had stored before

  CONCATENATE

  LOAD Distinct * FROM $(vQVD)\NEWFILE.qvd (QVD)

  where date(submitDateTime)>=date('$(dDate)'); 

Sometimes my historical data has the same month. For example I load February data, then concatenate historical data with February. I get double count for February.( instead of 10 , I get 20). Do you know how to avoid duplicate records?

Thanks,

Anonymous
Not applicable
Author

Thanks, I found out solution.