topic Re: converting into QVD from warehouse table in QlikView

converting into QVD from warehouse table

Anonymous — Thu, 16 Apr 2015 04:47:25 GMT

Hello experts,

I am new to qlikview.

I have one fact table and is having more than 20 million records. I am extracting data from it and converting into QVD. But it is taking more than 6 hours and not able to convert. Is there any way to convert this one with minimal time taking.

Regards,

Kumar

Re: converting into QVD from warehouse table

shambhub — Thu, 16 Apr 2015 06:38:11 GMT

Hi Kumar,

Make QVD file with where condition from the fact table.

Example: Year wise you can make QVD separate and at the end you concatenate all QVDs into one logical table in Qlikview.

Best Regards

Shambhu B

Re: converting into QVD from warehouse table

Anonymous — Thu, 16 Apr 2015 08:19:23 GMT

Actually this fact table is having only around 3months of data.

Regards

Re: converting into QVD from warehouse table

maleksafa — Thu, 16 Apr 2015 08:32:30 GMT

you need to check the query that you are using or the ETL process in the warehouse layer, normally reading data into a qvd should be fast.

but again it depend on how you are fetching this data (the query behind it)

Re: converting into QVD from warehouse table

Thu, 16 Apr 2015 08:37:04 GMT

Hi Kumar,

you'll also need to ensure you have enough resources on your server, as the data is loaded into memory before it is then saved down to the QVD, you'll need enough for the size of data load.

As others have mentioned, splitting QVD's down into Daily/Monthly files would help.

Joe

Re: converting into QVD from warehouse table

Gabriel — Thu, 16 Apr 2015 08:42:39 GMT

Hi,

Are you doing Transformation before storing your data into QVD?

As far as am aware storing data into QVD shouldn't take 6 hours. As mentioned above you probably want to check the query you're using to extract the data from source.

My suggestion would be, extract the data as is (raw) and store it into QVD, i.e.

DATA:

SELECT * //Or list of fields you want to extract

FROM

SOURCE;

STORE DATA INTO .......DATA.QVD(QVD);

Then you can load the raw data and transform it as you which.

Hope this helps

Re: converting into QVD from warehouse table

shambhub — Thu, 16 Apr 2015 09:47:40 GMT

Then find some other criteria to split QVD files and you need to check your server RAM also.

Have you created QVD file for the same fact table so far ?, if you are yes you can use incremental load concept to get future data.