Data load

[Anonymous]

Hi all,

Loading data in my app takes almost 25 min from script. How can I make it faster?

Thanks

In advance

[shilpan]

There are multiple factors that may impact load time.

1. Data volume

2.CPU and RAM power of the machine on which reload runs

3. Script itself may cause increased load time

4. # of sources and tables

[ChennaiahNallani]

Implement incremental load and avoid Synthetic tables and circuital loop.

[VishalWaghole]

1. Make sure that you will only load required fields.

2. Avoid synthetic keys and circular loop

3. Use joining keys in number data types, if you have in character then convert it by using autonumber function.

Regards,

Vishal

[ahmar811]

1-Avoid if condition when load data from data source

2-Don't put more than one where condition when load data directly from data source

3-Put condition checking or If condition after Table Resident

4-Avoid left join and Right join better to use Left keep and Right keep its faster as compare to Left and Right join

5-If you have multiple condition checking at the time of data load so try to avoid if function,you can also use pick function with wildmatch/match doing same things with faster processing .

6-Try to use mapping table and applymap in your script its faster than join.

7-Comments and exclude field which are not in used.

I hope above point is help you.

Regards

Ahmar

[Ivan_Bozov]

Also if you have historical data that is not changing you can save it as QVD file an load from that file instead of using Excel or DB as a source. QVD loads are 10-100 times faster.