Qlik Community

QlikView Scripting

Discussion Board for collaboration on QlikView Scripting.

Announcements
QlikView Fans! We’d love to hear from you.
Share your QlikView feedback with the product team… Click here to participate in our 5-minute survey.
Rules, plus terms and conditions, can be found here.
kumarreddy257
Contributor

converting into QVD from warehouse table

Hello experts,

I am new to qlikview.

I have one fact table and is having more than 20 million records. I am extracting data from it and converting into QVD. But it is taking more than 6 hours and not able to convert. Is there any way to convert this one with minimal time taking.

Regards,

Kumar

6 Replies
shambhub
Contributor

Re: converting into QVD from warehouse table

Hi Kumar,

Make QVD file with where condition from the fact table.

Example: Year wise you can make QVD separate and at the end you concatenate all QVDs into one logical table in Qlikview.

Best Regards

Shambhu B

kumarreddy257
Contributor

Re: converting into QVD from warehouse table

Actually this fact table is having only around 3months of data.

Regards

maleksafa
Valued Contributor

Re: converting into QVD from warehouse table

you need to check the query that you are using or the ETL process in the warehouse layer, normally reading data into a qvd should be fast.

but again it depend on how you are fetching this data (the query behind it)

Not applicable

Re: converting into QVD from warehouse table

Hi Kumar,

you'll also need to ensure you have enough resources on your server, as the data is loaded into memory before it is then saved down to the QVD, you'll need enough for the size of data load.

As others have mentioned, splitting QVD's down into Daily/Monthly files would help.

Joe

Gabriel
Valued Contributor II

Re: converting into QVD from warehouse table

Hi,

Are you doing Transformation before storing your data into QVD?

As far as am aware storing data into QVD shouldn't take 6 hours. As mentioned above you probably want to check the query you're using to extract the data from source.

My suggestion would be, extract the data as is (raw) and store it into QVD, i.e.

DATA:

SELECT * //Or list of fields you want to extract

FROM

SOURCE;

STORE DATA  INTO .......DATA.QVD(QVD);

Then you can load the raw data and transform it as you which.

Hope this helps


shambhub
Contributor

Re: converting into QVD from warehouse table

Then find some other criteria to split QVD files and you need to check your server RAM also.

Have you created QVD file for the same fact table so far ?, if you are yes you can use incremental load concept to get future data.

Community Browser