topic Re: How to optimize exporting data from database(Teradata)to qlikview in QlikView

How to optimize exporting data from database(Teradata)to qlikview

Mon, 26 Jan 2026 18:19:17 GMT

Hi,

i have a large database which having 20 crores records for every sales org, So i have 3500 sales org ID like this and i'm fetching data for each sales org wise and making Qvd's so for every Qvd for a sales org its taking 6-8 hours , So overall its taking 18 days to reload the full Application and sometimes Reload task has been failing because of more time taking and have to reload that again.

What should i do in that case? how can i optimize it?

Re: How to optimize exporting data from database(Teradata)to qlikview

avinashelite — Tue, 31 Mar 2015 10:28:16 GMT

Hi Yogita,

18days is a very big deal. I suggest you to try the direct discovery concept of Qlikview i.e no need to load all the data to in-memory instead bring only a required set and rest take it on fly.

Check with this link.

QlikTech and Teradata Partner To Deliver QlikView Direct Discovery for Big Data Analytics | Qlik

To reduce the reload time, try like this:

1.Instead of pulling all the data, pull only the required fields.

2.Do all the aggregation and store only necessary set of data.

3.If it taking so long time then divide the task between multiple qvw's that one to generate sales, one to organisation ,other etc

Re: How to optimize exporting data from database(Teradata)to qlikview

Tue, 31 Mar 2015 10:36:16 GMT

Hi ,

I'm using all these concepts .

i have divided task in to multiple tasks and all aggergation at the backend .

Re: How to optimize exporting data from database(Teradata)to qlikview

avinashelite — Tue, 31 Mar 2015 10:39:05 GMT

You need all the data from DB??? what's your RAM size?? because it also depends on the about of RAM size too.

share your script let me try to add any points on it.

Re: How to optimize exporting data from database(Teradata)to qlikview

Tue, 31 Mar 2015 10:59:35 GMT

what Direct Discovery is All about ?

Re: How to optimize exporting data from database(Teradata)to qlikview

avinashelite — Tue, 31 Mar 2015 11:12:26 GMT

Please refer this documents.

Re: How to optimize exporting data from database(Teradata)to qlikview

Tue, 31 Mar 2015 12:04:01 GMT

Thanks for the reply, i just have one doubt if direct discovery will fetch data directly from data source then application performance will go down as user selection response will go to data source everytime.

Re: How to optimize exporting data from database(Teradata)to qlikview

avinashelite — Tue, 31 Mar 2015 12:09:15 GMT

no, because you will pull all the dimension's into Qlikview(in-memory) just the measure calculation will be pushed to the DB so it won't consume much of the time.

NOTE:The dimension will get refreshed only when the application is refreshed.

Re: How to optimize exporting data from database(Teradata)to qlikview

Wed, 01 Apr 2015 08:38:01 GMT

Thanks Avinash , but is there any other way then Direct discovery.

Re: How to optimize exporting data from database(Teradata)to qlikview

avinashelite — Wed, 01 Apr 2015 09:30:21 GMT

we have some third party tools which will connect for the DB and generate qvd's for you e.g Qlikview Ex-presser try with that.