QVD join issue

[swati_rastogi27]

Hi ppl,

I am having issues in performing left out join with my QVDs.

Originally I had source data extract query that was pulling in data from database with 5-6 left outer joins.

Every table had millions of rows and eventually the execution was taking very long.

I split the individual table data into separate QVDs and then joined the QVDs together to create a query master set.

For small data set it is working very efficiently, but for large data sets the left outer join query fails with 'Script Error'

Any suggestions?

[Gysbert_Wassenaar]

Don't extract any data you will not be using in the document.

Don't extract all the data each time, but only the new and changed records. This is called incremental loading. If you search this site you can find many discussions on this topic.

Make the source database execute the joins and only retrieve the result that the database generates.

Make sure your machine has enough ram to execute the script successfully. Monitor ram using during the script execution to see if you're hitting the limit of available ram.