Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi Everyone,
I have one requirement like system has to do incremental process based on date in particular qvd
For Example. I have source file like
ID | Name | Date |
---|---|---|
1 | A | 01/01/2016 |
2 | B | 31/01/2016 |
3 | C | 01/02/2016 |
4 | D | 24/02/2016 |
5 | E | 01/03/2016 |
6 | F | 15/03/2016 |
Requirements:
1) Store the month wise data into particular month qvd like jan data into X_012016.qvd, Feb data into X_022016 and so on.
2) Now i need to write one for loop like system has to check the dates in oldest qvd first( in our case Jan month qvd) and then it should do incremental process( Insert or update) from DB and jan qvd should contains updated jan data. and it has to check for all qvd and hs to update data for all month qvds.
Kindly do the needful
Hi Rahul,
PFA for sample data.
I am using same data
Hi, maybe this will be heplfull, read https://community.qlik.com/servlet/JiveServlet/download/650953-136192/Incremental%20Load.docx
Hello Dada,
Please refer attached script as well as zip file. This will solve the purpose.
Thank you!
Rahul
Hi Rahul,
Attachment is missing.. can you attach the files
Hello Dada,
I could see the attachments. However, I have re-attached it herewith.
Also, refer below given modified script:
DateListTemp:
LOAD
Distinct LEFT(RIGHT(Date([Transaction date],'DD/MM/YYYY'),7),2)& RIGHT(Date([Transaction date],'DD/MM/YYYY'),4) AS MonthYear
FROM [Copy of incree load data.xlsx]
(ooxml, embedded labels, table is Sheet1);
NoConcatenate
DateList:
LOAD
MonthYear
Resident DateListTemp
ORDER BY 1;
Drop Table DateListTemp;
//EXIT SCRIPT;
FOR i=0 TO NoOfRows('DateList')-1
LET vMonthYear = PEEK('MonthYear', $(i), 'DateList');
LET vQVDPath = 'QVDs\'; // Set QVD Storage Directory
Sales:
LOAD
Region,
id,
product,
sales,
[Transaction date]
FROM [Copy of incree load data.xlsx]
(ooxml, embedded labels, table is Sheet1)
Where LEFT(RIGHT(Date([Transaction date],'DD/MM/YYYY'),7),2)& RIGHT(Date([Transaction date],'DD/MM/YYYY'),4) = $(vMonthYear);
// Check to see if this is the first reload. if it is, skip this step
if not isnull(QVDCreateTime('$(vQVDPath)Sales_$(vMonthYear).qvd')) THEN
Concatenate(Sales)
LOAD *
FROM $(vQVDPath)Sales_$(vMonthYear).qvd(qvd)
WHERE Not Exists(id);
end if
// If data exists within table, store to QVD.
if NoOfRows('Sales') > 0 THEN
STORE Sales INTO $(vQVDPath)Sales_$(vMonthYear).qvd;
Drop Table Sales;
end if
Next
Drop Table DateList;
Thank you!
Rahul
Hi Rahul,
Script is perfect to my requirement. Only thing is all values are loading from sources while doing incremental load for particular month. i have inserted new record for second load instead of loading only new record but it is loading all values. But it is really helpful and perfect script..
Thanks a lot
I am not able to mark it as correct answer.. Can you help me how to do that
Dear Rahul,
Would you please check my question and let me know your suggestion: