Saving a qvw in script to another file location or a better alternative?

[splitcore]

Hi guys..

Let me 1st explain what setup I have in mind.

Each day we receive new files containing data.

I have a qvw which contains this massive amount of data up to a certain date(Binary.qvw). Obviously reloading this model each day to only capture the new data is going to have to go look at all those files again and just won't do... (a single file contains about 250 000 lines and we have 4 years' worth)

So I have a second qvw(Incremental.qvw) which loads the 1st as binary, then has a incremental load to only load the newly placed files and concatenate it to the binary table loaded from the 1st qvw.

Here is where I am currently stuck... I then wish to save the Incremental.qvw which thanks to the incremental load now has the new data included, AS THE ORIGINAL Binary.qvw to overwrite it. So the next day the Binary.qvw will already have the previous day's data and only the Incremental.qvw would need to be run daily. But nowhere could I find a script command to store the qvw as: so that I could write it in at the end of my script.

I am sure I am missing some key theory here... Yes I could store the Binary.qvw as a qvd but the amount of data makes it difficult when I need to load it again (The qvd is about 15GB)...

Another thing that I thought I could maybe do was the following...

In my reload chain, I 1st run script of Binary.qvw, but in it's script I now load Incremental.qvw as a binary load containing the new data (Which was captured the previous day)! Next I run script for Incremental.qvw and 1st load Binary.qvw(Now containing the data up to the previous day) and then run the incremental load as usual to capture the new files and save it as itself. Basically a loop but I would need to run both qvw scripts...

Is there a better way to go about this? Is my logic flawed?

Or could this actually be the most resource friendly way to handle this massive amount of data?

Any advice or insights you can share would be greatly appreciated!

Kind Regards

Hendrik

[flipside]

Hi Hendrik,

The autogenerate part was just to create some data so probably you won't be needed this for your solution. It will be useful for you to know how it works, though. In my example my .qvw file has a table called Data with one field RowID. When I binary load the ,qvw it first loads this table, then that script adds 1 row to it. (If I had used, for example, Autogenerate 2, it would add 2 rows) When the file is saved it has one more row than before, and this repeats every time the script is run. I used RowNo() because this populates the field with the next incremented value - again just for demonstration purposes, you could use anything it would still get added.

Hope this helps.

flipside