Time Taken To load 20gb data into qlikview

[Anonymous]

Hi,

How much time it will take to load 20gb data from database to Qlikview,

Can someone help me to figure out this.

[Gysbert_Wassenaar]

That depends on lots of things. It's impossible to tell. If you have a machine with 4GB of ram then it will not load all that data ever. If you have a machine with an extremely slow harddisk it may take a very long time. If the 20GB of data are in separate xml files of 1KB average size it will take very long. If the 20GB is stored in one field in one qvd file on a super fast drive in a super fast machine with lots of fast ram then it can be read very fast. It all depends.

[Chanty4u]

chk dis

RAM and CPU Utilization%

CPU Utilization and Memory Utilization?

[Anonymous]

Thanks Gysbert, But if I have 126 GB RAM , and I am fetching a Table from DB which is of size 20gb,

how much approx. time it may take.

[oknotsen]

Strongly depends on what you do with the data, the columns, the distinct values, network connection, etc.

Data is calculated in rows, columns and distinct values, not in gb.

[Gysbert_Wassenaar · Accepted solution]

Impossible to tell. Is the table read from a file or from clay tablets? Is it read through a database connection or are the bits delivered using the RFC 2459 protocol * ? There are a thousand things that can influence the time.

* (ok, that's extremely unlikely, but still... as an example)

Try to make an estimate by loading the first 100.000 records:

OLEDB CONNECT  TO .... etc;

LET vStartTime = Now(1);

FIRST 100000

SQL SELECT * FROM MyTable;

LET vTotalTime = Interval(Now(1) - $(vStartTime),'hh:mm:ss.fff');

TRACE $(vTotalTime);

If you know how many records your 20GB contains you can calculated how long loading all the data will approximately take.

[Kushal_Chawda]

We can't say exactly how much time it will take to load the data. It depends on how fast is your database and how good is the network bandwidth between your QV server and database server. So try to load the data from machines with the different configurations and check which is the good machine to go for. To check the time to load the data you can try below.

ODBC .....

let vStartTime =now();

Table:

SQL SELECT *

FROM Source;

let vEndTime =now();

let vLoadingTime = interval('$(vEndTime )' - '$(vStartTime )','hh:mm:ss');

Check the value of variable vLoadingTime in input box

[Anonymous]

Thank you  so much Gysbert.

[Anonymous]

Thank you  so much.

[oknotsen]

Or simply check the time on the progress screen at the end.