topic upload only new csv in QlikView

upload only new csv

Wed, 25 Nov 2020 16:16:04 GMT

Hi to all,

I’m Cesare from italy and I’m new on qlikview.

I need support on new project I’m working on

I have a folder where weekly I deposit a csv file

I have built a simple script that allow me to upload all file present and store them in a QVD.

Something like this

Data:

LOAD
CONN_ID,
[CONCETTI ESTRATTI],
Cluster,
AgentGroup,
Team,

CHANNEL,

FROM

(txt, codepage is 1252, embedded labels, delimiter is ';', msq);

STORE Data INTO data.qvd (qvd);

The problem is when I deposit a new file, there are a lot of csv and the upload process is becoming too long.

I try to test some script in qlikview forum to upload only new file on qvd but I failed everytime.

Someone can help me about that?

Thanks

Cesare

Re: upload only new csv

Anonymous — Tue, 28 Mar 2017 16:02:49 GMT

Hey Cesare,

Welcome to Qlik,

Please use below like to do the incremental load.

Incremental Load of New Files Only

You can use filename related functions (find here) to traverse your file name and proceed.

Re: upload only new csv

annafuksa1 — Tue, 28 Mar 2017 16:04:01 GMT

add a date into your csv file and in a code add condition where for example date > WeekStart(today())

so only this week data

Re: upload only new csv

Mon, 03 Apr 2017 16:15:20 GMT

thank,

can you help me with a script that I can use for upload new csv data on QVD from a desktop folder.

Thank you

Re: upload only new csv

Anonymous — Tue, 04 Apr 2017 04:17:17 GMT

Can you provide the filename formats ?? or screenshot of the files in your folder?

Re: upload only new csv

Tue, 04 Apr 2017 13:07:59 GMT

Hi,

the file are

OLD FILE ON FORLDER

sales_01022017.csv

sales_01012017.csv

NEW FILE ON FOLDER

sales_01032017.csv

and the folder is 'C:\Users\baronices\Desktop\Sales'

I write successfully a script that load all files into qvd file.

When I deposit a new file (sales_01032017) I want to store the new file data on qvd without reload old files

// Load QVD

[DATA_TABLE]:

LOAD

Datarif,

Nome,

Amount

FROM

(qvd);

thanks very much

Re: upload only new csv

Anonymous — Wed, 05 Apr 2017 03:59:02 GMT

Do something like

set vFilePath = 'C:\Users\baronices\Desktop\Sales\';

For Each vFile in FileList ('$(vFilePath)sales_*.csv')

if Date(Date#(KeepChar($(vFile),'0123456789'),'MMDDYYYY'),'DD/MM/YYYY') > $(vLastReadFileDate) then

Data:

LOAD

CONN_ID,

[CONCETTI ESTRATTI],

Cluster,

AgentGroup,

Team,

CHANNEL

from $(vFile)

(txt, codepage is 1252, embedded labels, delimiter is ';', msq);

endif

LET vLastReadFileDate = Date(Date#(KeepChar($(vFile),'0123456789'),'MMDDYYYY'),'DD/MM/YYYY');

Re: upload only new csv

rahulpawarb — Wed, 05 Apr 2017 05:13:54 GMT

Hello Cesare,

Please refer below sample script logic:

If IsNull(QVDCreateTime('C:\Users\baronices\Desktop\data\QVD\'))
Then
Data:
LOAD 
  CONN_ID, 
  [CONCETTI ESTRATTI], 
  Cluster, 
  AgentGroup, 
  Team,
  CHANNEL
FROM

(txt, codepage is 1252, embedded labels, delimiter is ';', msq);

STORE Data INTO C:\Users\baronices\Desktop\data\QVD\data.qvd (qvd);


End if

Hope this will help.

Regards!

Rahul