topic Incremental reload issues in QlikView

Incremental reload issues

ankit_tikoo — Wed, 25 Nov 2020 16:16:04 GMT

Hi Folks

I am new to qlik and have a problem to fix. Need your help here.

So here it goes. I work on really meaty data and it shall become increasingly difficult to handle data if entire data keeps loading everytime the application is reloaded.

I need to keep incremental data. So for example, if I start from March 1, 2018, I want to load entire data from March 1 to March 13 once and later on every time the application loads the data which should be fetched should be for the next day which is March 14 only. This data should get stored along with the already stored data from March 1 to March 13 on which I need to run the visualization. This should keep happening automatically everyday so that any day when I open the application it should show data from March 1 to the previous day when I am opening the application.

Can someone please let me know the way out with syntax. Thanks in anticipation.

Re: Incremental reload issues

shiveshsingh — Wed, 14 Mar 2018 16:05:39 GMT

Take idea from below script

TEST:

LOAD Product_Id,

Product_Line,

Product_category,

Product_Subcategory

FROM

A.qvd

(qvd);

Max_ID:

load max(Product_Id) as Max_Id

Resident TEST;

Let Max_Id = Peek('Max_Id',0,MAX_ID);

DROP TABLE TEST;

New:

LOAD Product_Id,

Product_Line,

Product_category,

Product_Subcategory

FROM

Book1.xlsx

(ooxml, embedded labels, table is Sheet1) where Product_Id>$(Max_Id);

Concatenate

LOAD Product_Id,

Product_Line,

Product_category,

Product_Subcategory

FROM

A.qvd

(qvd);

Store New into B.qvd;

Just put your date filter in place of max(Product_ID)

Max_ID:

load max(Date) as Date

Resident TEST;

Schedule your task to run daily through QMC!

Re: Incremental reload issues

ankit_tikoo — Thu, 15 Mar 2018 12:44:23 GMT

Hi Tried the code, but facing some issues. I am attaching the files.

Test file is the original data to which new data must be added every day. Suppose if I do this whole process manually, then i dont need the last updated thing in queries.

Please check the attached files. I am using SQL queries. If you can help with the issue and share the same files using excel upload rather than sql, things will work.

Also, what if i have a qvw files instead of qvd.

I want to keep adding data is Test file everyday.

Please help

Re: Incremental reload issues

shiveshsingh — Fri, 16 Mar 2018 10:30:50 GMT

You are putting this code

Concatenate

LOAD * FROM Test.qvw;

This will not work, Can you share me the script that you are using to get the data as of now.

How are you managing the date filter?

I'll try to implement incremental load into it.