Qlik Community

QlikView Scripting

Discussion Board for collaboration on QlikView Scripting.

Announcements
QlikView Fans! We’d love to hear from you.
Share your QlikView feedback with the product team… Click here to participate in our 5-minute survey.
Rules, plus terms and conditions, can be found here.
Not applicable

Incremental Load hanging

Hi All,

I've got a lot of data in a SQL database, to speed up the loading time I decided it would make sense to load the data in chunks. Load some of the data, store it in a qvd file, load more data and the qvd file, then repeat, building up a qvd file with all but the most recent data.

When I have tried this the initial load takes 10-15 mins, this is saved to a qvd file fine. I then load the new data (takes the same sort of time), load the qvd file (loads much quicker), but then the CPU shows as maxing out for over an hour, without the Close button on the script execution process becoming available.

Any ideas as to why the process is not completing as normal when the data has finished loading?

Thanks,

Richard

Tags (1)
1 Solution

Accepted Solutions

Incremental Load hanging

My guess is that the new data load and the qvd have a different field list, causing the creation of two seperate tables and then massive key generation at the end of the script. Is that possible? If so, make the field list match or use the CONCATENATE keyword when loading the QVD.

-Rob

2 Replies
Not applicable

Incremental Load hanging

The incemental load logic may not be proper. For me, against a 25 million record full load, the incremental is taking only 10 minutes max. So I would suggest to review your load logic.

- Arun

Incremental Load hanging

My guess is that the new data load and the qvd have a different field list, causing the creation of two seperate tables and then massive key generation at the end of the script. Is that possible? If so, make the field list match or use the CONCATENATE keyword when loading the QVD.

-Rob

Community Browser