Stages, Data Verification & Running Process

[microwin88x]

Hello!

I have a project where there are 5 stages that filter data and join tables in order to gather the information I need to build my app, where each stage comes one after the other.

The thing is that those stages are updated weekly and all of them take around 3 hours in total.

Now, my question is the following: If I had to modify a particular stage in the future to add a new field o whatever it could be, I would need to reload all my stages again to get that information updated in my app, right? But the thing is that it would take something like 3 hours.

I wanted to ask you if there's any way to reduce the time of the whole process to test what I'm doing is okay and the data I'm showing is the one I need. How could I do this?

Thank you!!!

[#2 · Accepted solution]

Hi,

If you can verify your goals having only parts of the data loaded you can (in a test enviroment)

- try debug modus and check "First 10" to reduce data and loading time

or

- try to use First - clause of (all or some choosen) LOAD Statements. for ex:

First 100 LOAD ....  instead of only LOAD .... . Use "find/replace" if you have a lot of statements

BUT If you need to check the whole data you have to load it completely, even if it takes 30 hours.

HtH

Roland