Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Discover how organizations are unlocking new revenue streams: Watch here
cancel
Showing results for 
Search instead for 
Did you mean: 
userid128223
Creator
Creator

update with new time stamp.

Hi Everyone.

I have a SOURCE qvd with following type of records.

ID     Name     Amount      update_date

123     james     $300       Feb-01-2014    

127     John      $234         Feb-01-2014

547     Alex        $555       Feb-02-2014

New file from sales comes in daily basis with new and corrected records. Therefore I want to update any corrected records and append new records in above qvd.

for example today's sales file contains old & corrected records with update date.

ID     Name     Amount      update_date

123     james     $300       Feb-01-2014    // gets dropped as new entry from same ID but new date exist

123     james     $500       Feb-03-2014

127     John      $234        Feb-01-2014

547     Alex        $555      Feb-02-2014

432     Judy         $800     Feb-10-2014   // gets added as new entry.

My source should get updated with all the new records + all the updated records with latest update date.

Final results.

ID     Name     Amount      update_date

123     james     $500        Feb-03-2014

127     John       $234        Feb-01-2014

547     Alex       $555        Feb-02-2014

432     Judy       $800        Feb-10-2014

thanks

18 Replies
swuehl
MVP
MVP

Either try to load your daily data sorted by ID and Date desc, then use the where not exists (ID) clause,

or load your daily data:


DAILY:

LOAD * INLINE [

ID,     Name,     Amount,      update_date

123,     james,     $300  ,     Feb-01-2014

123,     james,     $500  ,     Feb-03-2014

127,     John ,     $234   ,     Feb-01-2014

547,     Alex ,       $555 ,     Feb-02-2014

432,     Judy ,        $800 ,    Feb-10-2014 

];

and then

LOAD *, ID as TmpID Resident DAILY where not exists(TmpID, ID) order by ID, update_date desc ;

drop table DAILY;


drop field TmpID;

Not applicable

Ufff

I'm not sure but you can try using an applymap with a key (ID&'_'&update-date) with the old information and put a flag to remove this rows:

Map_Remove:

Mapping

Load
       ID & '_' & update-date as Remove,

       0 as flagtoremove

from oldinformation;

Sales:

Load

       ID,

       Name,

       Amount,

       update_date,

       applymap('Map_remove', ID & '_' & update-date,1) as flag

from newinformation

where applymap('Map_remove', ID & '_' & update-date,1) = 0;

Above keep only actual information delete old information in second source.

Best regards.

userid128223
Creator
Creator
Author

Hi Swuehl

I will try this and let you know, however can you please explain this code.

LOAD *, ID as TmpID Resident DAILY where not exists(TmpID, ID) order by ID, update_date desc ;



how does it work, whats the logic.


thanks

userid128223
Creator
Creator
Author

HI Swuehl

so you are saying if I sort the data by ID and Date desc and then use not exist it will only pick the first instance of record and that way get the latest update.

second question how do you sort by ID and Date desc

userid128223
Creator
Creator
Author

Can you please explain what this code does.

where Peek(ID) <> ID

order by ID, update_date desc;

userid128223
Creator
Creator
Author

I have yet to try your solution. But i will give it a try.


where not exists(ID,ID);   // what does second parameter do.



maxgro
MVP
MVP

Final:

NoConcatenate load  *

Resident Source                         // 1*

where Peek(ID) <> ID                  // 2*      

order by ID, update_date desc;    // 3*


1*                                  read from a previous loaded table (resident=already in memory)

2* and 3* :

order by:                        I want to read the record sorted by ID and, for same ID, update_desc descending

where Peek(ID) <> ID      filter: discards rows if the ID is the same of previus read record


example

ID     update_date           Peek(ID)                      

1     31-12-2010                               keep              

1     10-10-2009               1                          

1     1-1-2007                   1                         

2     1-1-2010                   1              keep

2     1-1-2009                   2   

2     1-1-2008                   2   

3     12-10-2014                2              keep

4     10-10-2007                3              keep


result: the 4 rows in bold

from online help

peek(fieldname [ , row [ , tablename ] ] )

Returns the contents of the fieldname in the record specified by row in the internal table tablename. Data are fetched from the associative QlikView database.

Fieldname must be given as a string (e.g. a quoted literal).

Row must be an integer. 0 denotes the first record, 1 the second and so on. Negative numbers indicate order from the end of the table. -1 denotes the last record read.

If no row is stated, -1 is assumed

userid128223
Creator
Creator
Author

Hi Massimo

What if my data has 2 dates and the uniq key is made of 3 fields example:

ID&'|'&Name&'|'CreatedDate as UniqID.

Where CreatedDate is used only to find uniq combination but update date will be used to determine what records gets purged or appended.

How do you handle that scenerio?

Source:LOAD * INLINE ; ConcatenateLOAD * INLINE ; Final:NoConcatenate load*Resident Sourcewhere Peek(ID) <> IDorder by ID, update_date desc; drop table Source;

 

RESULT

IDNameAmountupdate_date123james500feb-03-2014127John234feb-01-2014432Judy800feb-10-2014547Alex555feb-02-2014

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

userid128223
Creator
Creator
Author

I will mark 2 as correct answer, however I will open up new thread as my requirement is now different then what I posted here.

thanks everyone for help.