topic Re: How to prevent loading the same file twice in Qlik Learning Discussions

How to prevent loading the same file twice

Anonymous — Thu, 04 Sep 2014 09:21:07 GMT

Hi,

I'm working on a Stock analysis tool for my first qlikview project.

The only issue I'm struggling with is to prevent the script from importing the same "changed_stock.csv" more often than once.

Could you please give me an advice how to solve this?

I thought about deleting the file after import, or adding an unique id to the csv and import with a where not exists.

//Loading price table
LOAD @1 as ArtNr,
     @2 as ArtikelBezeichnung,
     @7 as Bewertungspreis,
     @8 as PEinheit
FROM
[\\s-m3qv\flatfiles$\CSB\Preise 2013.xls]
(biff, no labels, header is 9 lines, table is [Preise 2013$]);

//Loading Stock table including historical data
Stock:
LOAD ArtNr,
     LosNr,
     Lager,
     Datum,
     Menge,
     Einheit
FROM
[\\*****\Stock.qvd]
(qvd);

//Loading changed Stock data into the Stock table
StockIncremenal:
Concatenate
LOAD [Art.-Nr.] as ArtNr,
     LosNr,
     Lager,
     Datum,
     Menge,
     Einheit,
     MHD
FROM
[\\*****\stock_changed.csv]
(txt, codepage is 1252, embedded labels, delimiter is ';')
;

//overwrite the stock qvd with the merged data
Store Stock into '\\*****\Stock.qvd' (qvd);

Kind Regards

Re: How to prevent loading the same file twice

Gysbert_Wassenaar — Thu, 04 Sep 2014 10:35:18 GMT

Perhaps this blog post helps: qlikviewcookbook.com/2012/01/incremental-load-using-qlikview-components/

Re: How to prevent loading the same file twice

Anonymous — Mon, 15 Sep 2014 11:48:32 GMT

Thanks for the advice, but my source table and the update table doesn't meet the following attributes.

To utilize Incremental Load a source table must have both of the following attributes:

A unique identifier — a Primary Key — for each row.

A “Modification” column that identifies when a row was added or updated. The column type may be a Date, Datetime or ascending Revision number.

Is there any way to add these primary keys & modification collumns to my source table in qlikview? Its about 4million entrys long, so I cant edit it in excel.

Re: How to prevent loading the same file twice

deepakqlikview_123 — Mon, 15 Sep 2014 12:08:45 GMT

use Recno/rowno function for generating unique keys,guys plz correct me if I am wrong.

Thanks

Re: How to prevent loading the same file twice

Agis-Kalogiannis — Mon, 15 Sep 2014 12:19:09 GMT

I think you are right there Deepak

So if you add a line in your script, where you load the stock_changed.csv like the following

RowNo() AS StockID

you immediately create your unique identified when you load the tables

Re: How to prevent loading the same file twice

Anonymous — Tue, 16 Sep 2014 08:39:20 GMT

Thanks a lot guys! RowNo worked very well!