Slow load from Paradox

Tue, 15 Jun 2010 14:50:29 GMT

Hi guys - hope you can help

I've got some table files (Paradox) to bring into QlikView as a test for a potential client but they take ages to load.

One of the larger files takes ages to load using the Microsoft / Paradox odbc driver (659k records). It starts off fast but starts to slow down as more records are read as follows.

Records

Records Imported	Time
43k	10sec
56k	20 sec
80k	40 sec
118k	90 sec
299k	9 min
346k	14 min
556k	42 min
659k	1 hr & 1min

I've run a conversion (to csv) and to import the csv version into QV takes only 3 mins in total so I guess it's a problem with the driver.

I've tried just slecting a few fields but it only makes a slight difference.

Any ideas ??

Slow load from Paradox

Oleg_Troyansky — Tue, 15 Jun 2010 17:17:13 GMT

It certainly looks like a Driver issue, so I'd start from talking to either Microsoft or someone who's specializing on Paradox (if those people exist anymore...)

If no external help is available, here is what I'd try:

1. If possible, chop the load into many smaller loads - for example, if this is transactional data, you could load each month separately (in a loop) and let QlikView automatically concatenate the results. It looks like 15 loads of 43 K will only take 15x10 = 150 sec. So, the smaller the chunks, the better.

2. If you need to read this file on a regular basis, try to organize incremental load logic and only read a little bit of fresh data every time - then the slow down won't be significant.

good luck!

topic Slow load from Paradox in QlikView

Slow load from Paradox

Slow load from Paradox