Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
jason_nicholas
Creator II
Creator II

Using QVD to load prior-year data

I am having trouble loading QVD files containing prior year data. When I add these to the load script, the load time increases exponentially. My application is not particularly complex as these things go, and a reload generally takes 5-7 seconds. With the addition of the QVD loads, the execution time jumps to somewhere over 20 minutes. This seems counterintuitive to what QVDs are supposed to do, so I assume I am making a mistake.

My application is loading source data for the year 2017. It runs a number of transformations, calculations, and all of the other stuff QV does to generate an internal table. With just 2017 data, this functionality works well. The data loads and the charts populate. However, when I try to load 2016 data through QVD files, the load script freezes and takes an extended time to execute.

The QVD files are created by taking a copy of my 2017 application and replacing the source data. I then store the four internal tables created by my load script into QVD files, to be loaded into the 2017 application. To be clear, my QVD files are in the format of the  internal table generated by my load script, and not in the format of the source data.

My expectation is that by loading those QVD files into my original application, the tables should concatenate and create a single table, complete with 2016 and 2017 data. Is this correct? Or do the tables not properly join because the QVDs are formatted like the resulting table and not like the source?

I have tried CONCATENATE and JOIN statements, and I have also tried NoCONCATENATE to try to manage it as separate tables. I've placed the QVD loads in line with the rest, and I have tried posting them to their own tab after the completed load of the primary table. Each of these have had the same result of an extended reload time.

Hopefully, this is enough information to elicit a solution. I am unable to upload an example file, so descriptions and script examples are all I have to work with here. Thanks for the help.

1 Solution

Accepted Solutions
jason_nicholas
Creator II
Creator II
Author

EDIT: The problem was one of my own making. After waiting for a 43 minute load time, I tried to re-order the script and came to a solution.

It was an issue with my load order. I needed to load the QVD file with CONCATENATE directly after the script that creates the same table for current data. When I tried this previously, there was a completely separate problem preventing it from working. That problem has subsequently been fixed, and the re-order now works properly.

View solution in original post

1 Reply
jason_nicholas
Creator II
Creator II
Author

EDIT: The problem was one of my own making. After waiting for a 43 minute load time, I tried to re-order the script and came to a solution.

It was an issue with my load order. I needed to load the QVD file with CONCATENATE directly after the script that creates the same table for current data. When I tried this previously, there was a completely separate problem preventing it from working. That problem has subsequently been fixed, and the re-order now works properly.