which database to use with PE and big datasets

[#1]

Dear all,

until now I used Qlikview importing large csv files and to make dashboards, analytics based on that data.

As, for example, I make a weather analytics of the last 30 years, the csv files are getting quite big and sometimes I yet got the error - running out of memory...

So I was wondering if it makes sense to set up (e.g.) a sql database?

Do you have any further documentation on how to work with "stand alone" databases?

Kind regards and thank you

Thorsten

[petter]

You mean running out of memory while loading the data from the CSV in a load script?

If that is the case - the loading of the data into QlikView in-memory through a SQL-database will not make a difference. Using a SQL-database as a source does not help on memory. I

f you on the other hand use Direct Dicovery for your fact table then using a SQL-database will help. But it comes with some trade-offs and limitations. So I am pretty sure that doing the following two things first will be most useful for you:

1)  Look at how the data you bring in can be made less memory consuming.

2) Buy more memory for your computer

[HirisH_V7]

Hi,

• Are you converting you data into QVD and Making use of them in your App.

• Try using incremental load beacause .I hope the old data of weather will not have a change.This may decrease your Reload time and only new records will be appended.

• CSV's -the number of records is limited to 1048576 rows.

• I think in your csv export you can create a export based on monthly or quaterly CSV such that the records will be lesser than the limitation.

HTH,

Hirish